Daniel James Harper
Enthusiastic, passionate developer with 11 years of software development experience in enterprise and personal projects, including data engineering and web applications. I am seeking a role in a creative, forward thinking company that offers personal development and challenging work.
Experience
Software Engineer, Cloudflare
March 2020 -
London, UK
Senior Software Engineer, BBC
June 2018 - February 2020
London, UK
Main responsibilities
Developing and maintaining “big data” analytics systems that help the BBC manage costs and deliver the best experiences to the iPlayer consuming public.
Key technologies
Java, Presto, Beam/Flink, Amazon Web Services (Kinesis, Lambda, EMR, S3, DynamoDB, EC2, Aurora, SQS, Athena), Hadoop, Python, Node.js, Parquet file format
Recent projects
Developing a stream processing system to collect anonymised A/V event data from iPlayer clients and help visualise this to interested parties through a web based visualisation tool. Written in Java 8, using technologies such as Kinesis, Apache Beam and Athena to process, aggregate and query the data.
Developing a batch based log processing system to help the BBC monitor A/V distribution over the internet. This consumes logs from a variety of Content Delivery Networks (CDNs), enriches them with additional information, then aggregates the data for use by a web based visualisation tool. This system ingests 20 billion rows a day, and has results available for analysis within minutes of the source arriving. Written in Java 8 and makes use of cloud technologies such as AWS Lambda, S3, Aurora, EC2 and DynamoDB. This replaced a batch processing system that used Hadoop on AWS.
Developing a stream processing system that the BBC use to monitor live streams from the iPlayer, and alert operations teams when something has gone wrong. This system aggregates 4000-7000 events per second and can alert teams within one minute of detected aberrations occurring. Written in Node.js and makes use of cloud technologies such as AWS Kinesis and Lambda.
Other achievements
Published a blog post on the BBC Internet blog about a project I worked on within the team see https://www.bbc.co.uk/blogs/internet/entries/04a4b988-504f-4a87-aadd-2d8db57c3677
Software Engineer, BBC
July 2014 - May 2018
London, UK
Developing and maintaining “big data” analytics systems that help the BBC manage costs and deliver the best experiences to the iPlayer consuming public.
Applications Developer, Sony Computer Entertainment Europe
August 2010 - July 2014
London, UK
Main responsibilities
Developing and maintaining enterprise software for the European territories of the SCE group, using a Java/JEE based development stack.
Key technologies
Java, J2EE, Spring, Spring MVC, Oracle 11g, Perforce, Weblogic
Summary
My role as a developer was to develop and maintain a suite of internal facing applications based on the needs of the business.
Notable projects included: -
- Taking on Lead Developer role on a project to help the business to increase revenue on voucher sales to retailers. This involved designing and building an application using Spring MVC to help retailers engage earlier on upcoming products and manage orders. Since going live the business put over €12m worth of vouchers through the system
- Designing and implementing a new logistics application using Spring MVC and MyBatis frameworks to manage shipments and truck movements. This was used extensively through the run up to the Playstation 4 launch.
Applications Developer, Atos
August 2008 - July 2010
Nottingham, UK
Main Responsibilities
Developing and maintaining applications for company clients within the public and private sector using Java/J2EE, including: -
- Natural England/DEFRA
- Travelex
- Network Rail
Education
University of Lincoln
September 2005 - July 2008
BSc (Hons) Games Computing
First Class Honours (1:1)
Dissertation (Final year project)
Is a steering orientated driving agent more effective than those that are pre-trained?
This involved developing an AI system that allows a vehicle to drive around a given track unaided by any prior training. Written C++ and uses the OpenGL API.
Lakeview College Sixth Form Centre
September 2003 - July 2005
A-Levels
Computing (A), Media Studies (C), Geography (D)
Courses
BBC
- AWS Essentials
- Architecting on AWS
- Big Data on AWS
- Introduction to Google Cloud
- Distributed Systems (Caches)
- SSL/TLS and Internet PKI
- Effective Use of Kanban
Sony Computer Entertainment Europe
- Java Enterprise Edition
Atos
- Introduction to PL/SQL
Technical Attributes
Languages I enjoy using: Java, Go
Operating Systems I’m comfortable with: OSX, Ubuntu Linux, CentOS
Datastores I’ve had experience with: DynamoDB, S3 (with Hadoop/Hive, Athena, Presto), AWS Aurora, Oracle 11g, Postgres 9.3
Cloud/Web services: Amazon Web Services (Kinesis, Lambda, EMR, S3), Heroku
About Me
Outside of programming and technical activities, I’d like to emphasise my other interests and pursuits, including:
- A life long passion for videogames, born from the early days of the Amiga 1200.
- Writing, some of my blog posts have reached the front page of Hacker News (see https://djharper.dev/)
- Being an avid reader of a wide variety of books, both nonfiction and fiction. My favourite book of all time is 1984 by George Orwell.
- Listening to live radio and podcasts, mainly themed around comedy, documentaries, technology and video games.
- Winner of “Best application” award at the Guardian “SXSW Hackday” 2011.
Personal Projects
I have a number of projects that I enjoy working on in my spare time. These include:
- A simulation of a simple computer, from the logic gate level upwards
- A gameboy color emulator written in Go
- A website that helps people determine mortgage affordability
Contact Me
See my Stack Overflow page for more information https://stackoverflow.com/users/story/199278
