DevOps Engineer at DataSift
Reading, Berkshire, United Kingdom
🇬🇧 (Posted Aug 22 2015)
About the company
We are DataSift. The hottest Big Data platform company in the world.
And we mean - really really Big Data. Our platform processes billions of interactions from social networks, blogs and news sites every day.
We are on the cutting edge of how human-generated data (social, blog, news and business data) can be used by brands and application developers to get to know their customers better and build new products faster.
DataSift is rapidly growing and currently have more than 160 employees worldwide with offices in San Francisco, New York, Los Angeles, London and Reading, UK and Canada. We work with the biggest brands such as Bloomberg, CBS Interactive and Dell as well as thousands of application developers across a wide variety of industries including retail, telecommunications, high tech, consumer packaged goods and digital agencies. We are well-funded by amazing investors, including Scale Venture Partners, Upfront Ventures and IA Ventures.
We plan on doubling in size over the next year, so we need your help. If you’re up to the challenge, keep reading and apply.
As a DataSift engineer, you will have the opportunity to work on several different technology stacks, ranging from PHP/Web Development to Java/Data Warehousing, to C++/Processing Engine. Some of the challenges we get to work on daily range from Web Scaling to Distributed Systems to Big Data. Pretty much every system at DataSift works within our Service Oriented Architecture, with REST and ZMQ predominantly used for communication.
What you will be doing:
Deploying, maintaining and scaling DataSift's infrastructure and application
Work using our "monitor everything" principle, gathering metrics and taking appropriate action
Working closely with our systems automation framework, Chef
Diagnose issues with hardware, software, network and applications
Taking part in our 24x7 on-call rotation
At DataSift we process over 2 Billion tweets, posts and updates each day. This requires an incredibly solid and scalable infrastructure. To support this our tech stack looks like this;
Service-oriented architecture, communicating over ZMQ and REST
500+ CentOS physical servers in production
All servers entirely managed by Chef and Rundeck
Hadoop for bulk data storage
Arista switches and Cisco routers
Graphite and Riemann used for all levels of monitoring
A mix of open-source and in-house software, written in the right language for the job, ranging from C++, Golang and Scala to PHP and Python.
Extensive development and testing environment infrastructure running on Openstack
Skills & requirements
Passionate about Linux and Open Source technology and you strive to use up-to-date tools and techniques
Very comfortable working on the Linux command line and have a broad knowledge of the surrounding technologies
You have a 2:1 or higher Computer Science, Software Engineering or related degree or can demonstrate additional relevant experience
Significantly experienced working in an Operations or System Administration role
Practical knowledge of shell scripting and at least one of our primary languages (C++, Go, PHP, or Scala), or in Ruby, Python or Java
Experienced in monitoring systems or services, ideally using software such as Graphite, Riemann or collectd
Familiar with systems automation tools, such as Chef, Puppet or Ansible
You have a good understanding and passion for working with and optimising distributed systems
You have the ability to collaborate with and enjoy a small, intense team of engineers and business people
Strong verbal and written communication skills
Readiness to take responsibility and go the extra yard to hit a deadline
Exposure to modern distributed systems software such as Hadoop, ZMQ, Elasticsearch, kafka or zookeeper.
Instructions how to apply
see the website
[ job website
Let them know you found the job via https://www.golangprojects.com
(Companies love to know recruiting strategies that work)