Senior Apache NiFi developer required to gather requirements, to design, development and production roll out and support for various systems. (157559)

Our GTA based, Municipal Government client requires the services of an Apache NiFi developer to gather requirements, to design, development and production roll out and support for various systems. gather requirements, to design, development and production roll out and support for various systems.

MUST HAVE:

A senior NiFi developer should have a robust skillset to design, develop, and maintain complex data integration workflows using Apache NiFi.

• Expertise in Apache NiFi
• Deep understanding of NiFi architecture, components (processors, controller services, flowfiles, etc.), and dataflow concepts.
• Proficiency in creating, managing, and optimizing NiFi dataflows for high throughput, low-latency data processing.
• Familiarity with NiFi Expression Language for dynamic configuration and routing.
• Data Integration and ETL Strong experience in designing and implementing ETL (Extract, Transform, Load) pipelines. Ability to integrate diverse data sources (databases, APIs, file systems, message queues, etc.) and sinks. Understanding of data formats (JSON, XML, CSV, Avro, Parquet, etc.) and schema management.
• Programming and Scripting Proficiency in Java, Groovy, Python for automation and custom logic. Familiarity with REST API integration for interacting with NiFi’s API for automation and monitoring. o Big Data Ecosystem Knowledge of integration with big data tools like Hadoop, Kafka, Spark, Hive, or HBase. Experience with message brokers (e.g., Kafka, RabbitMQ) for real-time data streaming. Understanding of cloud-based data platforms (AWS, Azure, Google Cloud) and their integration with NiFi.
• Database and Storage Systems Proficiency in working with relational databases (e.g., MySQL, PostgreSQL, Oracle) and NoSQL databases (e.g., MongoDB, Cassandra). Familiarity with cloud storage solutions (e.g., S3, Azure Data Lake, Google Cloud Storage). Knowledge of SQL for querying and transforming data. o DevOps and CI/CD Experience with containerization (Docker, Kubernetes) for deploying NiFi in modern environments. Familiarity with CI/CD pipelines (Jenkins, GitLab CI, etc.) for automating NiFi flow deployments. Knowledge of infrastructure as code (IaC) tools like Terraform or Ansible for environment setup.
• Monitoring and Performance Tuning Expertise in monitoring NiFi performance using tools like NiFi’s built-in monitoring, Grafana, or Prometheus. Ability to optimize dataflows for performance, scalability, and resource efficiency. Experience with logging, debugging, and troubleshooting NiFi issues.
• Security Understanding of NiFi security features, including SSL/TLS, authentication (LDAP, Kerberos, OAuth), and authorization (role-based access control). Knowledge of data encryption, secure data transfer, and compliance with standards like GDPR or HIPAA. Soft Skills o Problem-Solving Ability to analyze complex data integration challenges and design efficient solutions. Strong debugging and troubleshooting skills to resolve pipeline issues.
• Data Engineering Best Practices Understanding of data governance, lineage, and metadata management. Familiarity with data quality frameworks and ensuring data integrity in pipelines. o Industry-Specific Knowledge