Senior Software Engineer, Storage (Python)
Business Area:
Engineering
Seniority Level:
Mid-Senior level
Job Description:
At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we’re the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.
The Data Platform Pillar is the bedrock of Cloudera’s technology, where we design and build the core components that let our customers store, manage, and process data with unmatched scalability, security, and performance.
Cloudera is looking for an exceptional and passionate software engineer with some distributed systems background to join the Storage Engineering team focused on building Apache Ozone. The Storage team is responsible for primary storage and storage access layers, which are core to the platform. Apache Ozone (Apache Ozone) provides a massively scalable distributed object store with a distributed file system interface. Ozone is designed to scale to tens of billions of files and blocks, and overcome the limitations of Hadoop Distributed File System (HDFS), namely, millions of small files and managing a huge number of datanodes.
Ozone is one of the fastest-growing products inside CDP in terms of customer adoption and expansion revenue. Opportunity to join the team that created and wrote most of the HDFS code and make a huge impact on the big data and cloud computing industry.
As a Senior Software Engineer, you will…
Review, simplify, and rationalise already existing test cases and our internal testing framework code.
Prepare and implement test plans for newly developed features, and be part of the design process to ensure that testability is a concern from the beginning of the feature development.
Review and work on the different levels of testing within open source projects.
Work with our internal teams to integrate different layers of tests into our internal workflows related to development and supporting our customers.
Will be responsible for continuously increasing the quality of the storage layer within Cloudera’s Data Platform.
Develop an understanding of popular open source projects of Apache Hadoop; hyperscale cloud platforms like AWS, Azure and Container technologies like Kubernetes and Docker.
We are excited if you have…
Strong programming skills in one or more of the following languages: Python or Java, or JavaScript
Ability to design, build and maintain automated testing frameworks, tools, and automated test suites, in Python (pytest), preferred or Java (TestNG/JUnit)
Sound knowledge of test methodologies, including the creation of test cases and test plans
Good Debugging skills, esp. involving distributed systems, preferably on Linux
Ability to work closely with the Engineering teams and come up with test scenarios for new features, involving Big Data technologies
Working knowledge in storage systems and experience in developing and executing comprehensive storage testing strategies, evaluating functional, performance, scalability, stress, integrity, and security aspects of storage systems will be considered a strong asset
Ability to design and maintain CI/CD pipelines for enabling fast-paced, low-touch releases of our product
Ability to work effectively both independently and as part of a team.
You may also have…
Some background in a distributed storage system, including file systems, database storage internals, NoSQL storage, or distributed hash tables
Some background in performance tuning, identifying performance bottlenecks, and implementing performance optimisations
Understanding of the Apache Big Data ecosystem and experience in systems software, including file systems
Recognised contributions to open source projects
Experience using projects such as Hive, Pig, MapReduce, HBase, etc., is a big plus
Good Understanding of storage development, RAFT replication framework, or equivalent distributed consensus frameworks
Knowledge of Public Clouds (AWS/Azure) and/or Container Technologies (Docker, Kubernetes) is a plus
Why this role matters:
You will tackle complex distributed systems challenges, crafting the foundational software for the control and data planes that power CDP and keep it running at massive scale. Working at the forefront of hybrid and multi-cloud technology, you will empower data scientists, engineers, and analysts with the tools and infrastructure they need for advanced analytics and modelling.
Collaboration is key, you will work alongside brilliant minds across product, data science, and engineering to drive innovation, standardise best practices, and shape the future of enterprise AI and data platforms. This is your chance to build the future of data and see your work make a global impact.
What you can expect from us:
Generous PTO Policy
Support work life balance with Unplugged Days
Flexible WFH Policy
Mental & Physical Wellness programs
Phone and Internet Reimbursement program
Access to Continued Career Development
Comprehensive Benefits and Competitive Packages
Employee Resource Groups
EEO/VEVRAA
#LI-ZC1
#LI-REMOTE
