Golang / Go Job: DevOps Engineer

DevOps Engineer at DataSift

🇬🇧 United Kingdom › Reading, Berkshire   (Posted Aug 22 2015)

About the company
We are DataSift. The hottest Big Data platform company in the world.

And we mean - really really Big Data. Our platform processes billions of interactions from social networks, blogs and news sites every day.

We are on the cutting edge of how human-generated data (social, blog, news and business data) can be used by brands and application developers to get to know their customers better and build new products faster.
DataSift is rapidly growing and currently have more than 160 employees worldwide with offices in San Francisco, New York, Los Angeles, London and Reading, UK and Canada. We work with the biggest brands such as Bloomberg, CBS Interactive and Dell as well as thousands of application developers across a wide variety of industries including retail, telecommunications, high tech, consumer packaged goods and digital agencies. We are well-funded by amazing investors, including Scale Venture Partners, Upfront Ventures and IA Ventures.

Job position
Permanent

Job description
We plan on doubling in size over the next year, so we need your help. If you’re up to the challenge, keep reading and apply.

As a DataSift engineer, you will have the opportunity to work on several different technology stacks, ranging from PHP/Web Development to Java/Data Warehousing, to C++/Processing Engine. Some of the challenges we get to work on daily range from Web Scaling to Distributed Systems to Big Data. Pretty much every system at DataSift works within our Service Oriented Architecture, with REST and ZMQ predominantly used for communication.

What you will be doing:

Deploying, maintaining and scaling DataSift's infrastructure and application

Work using our "monitor everything" principle, gathering metrics and taking appropriate action

Working closely with our systems automation framework, Chef

Diagnose issues with hardware, software, network and applications

Taking part in our 24x7 on-call rotation

Our stack:

At DataSift we process over 2 Billion tweets, posts and updates each day. This requires an incredibly solid and scalable infrastructure. To support this our tech stack looks like this;

Service-oriented architecture, communicating over ZMQ and REST

500+ CentOS physical servers in production

All servers entirely managed by Chef and Rundeck

Hadoop for bulk data storage

Arista switches and Cisco routers

Graphite and Riemann used for all levels of monitoring

A mix of open-source and in-house software, written in the right language for the job, ranging from C++, Golang and Scala to PHP and Python.

Extensive development and testing environment infrastructure running on Openstack

Skills & requirements
About you:

Passionate about Linux and Open Source technology and you strive to use up-to-date tools and techniques

Very comfortable working on the Linux command line and have a broad knowledge of the surrounding technologies

You have a 2:1 or higher Computer Science, Software Engineering or related degree or can demonstrate additional relevant experience

Significantly experienced working in an Operations or System Administration role

Practical knowledge of shell scripting and at least one of our primary languages (C++, Go, PHP, or Scala), or in Ruby, Python or Java

Experienced in monitoring systems or services, ideally using software such as Graphite, Riemann or collectd

Familiar with systems automation tools, such as Chef, Puppet or Ansible

You have a good understanding and passion for working with and optimising distributed systems

You have the ability to collaborate with and enjoy a small, intense team of engineers and business people

Strong verbal and written communication skills

Readiness to take responsibility and go the extra yard to hit a deadline

Useful extras:

Exposure to modern distributed systems software such as Hadoop, ZMQ, Elasticsearch, kafka or zookeeper.

Apply now!