Senior Python developer to build an enterprise data warehouse.

Our valued public sector client requires a Senior Python developer to build an enterprise data warehouse.

They require an expert to perform application development, enhancements, and maintenance for various applications, components, and platforms. These may include data and analytics platform (data lake), data ingestion components, self-serve analytics, and data visualization tools.

Tasks:

• Foster a collaborative and innovative development environment.
• Python scripting for automation of storage-related tasks, configuration management, and system monitoring
• Utilize Azure Synapse to distribute the workload (performance management)
• Manage workload using Azure Synapse for a data lake project.
• Develop and maintain Python scripts for efficient data manipulation, migration, and integration with storage systems.
• Develop spark applications in python (PySpark) on distributed environment
• Creating Spark clusters and configuring high concurrency clusters
• Creating net new code and enhancement of existing code
• Create and maintain documentation for compliance and knowledge transfer
• Design for scalability and optimize code for peak performance.
• Develop test cases for unit and integration tests

Must-Have Skills:

• 5+ years experience with Python core programming
• 5+ years experience with Python libraries
• 5+ years Azure Synapse
• 5+ years experience with Spark and PySpark programming and performance tuning

Nice-to-Have Skills:

• 5+ years experience with Python unit testing libraries such as Pytest, unit test, nose