To Apply for this Job Click Here

Location: Remote EST hours

Duration: Ongoing multi year contract

Rate: 50-58/hr

Job Summary: The Data Engineer is responsible for designing, developing, and maintaining data pipelines and infrastructure to support data-driven decision-making within the organization. This role requires strong proficiency in Python, SQL, cloud technologies, and Kubernetes to ensure the efficient processing, storage, and retrieval of data.

Key Responsibilities:

1. Data Pipeline Development:

   • Design, build, and maintain scalable and reliable data pipelines using Python and SQL.
   • Develop ETL (Extract, Transform, Load) processes to integrate data from various sources into data warehouses and databases.
   • Ensure data quality and consistency across different data sources and systems.

2. Cloud Data Infrastructure:

   • Implement and manage data storage and processing solutions on cloud platforms (e.g., AWS, Azure, Google Cloud).
   • Utilize cloud data services such as BigQuery, Snowflake, Redshift, or similar to store and analyze large datasets.
   • Configure and manage cloud resources for optimal performance and cost-efficiency.

3. Containerization and Orchestration:

   • Deploy and manage data applications and services using Docker and Kubernetes.
   • Develop and maintain Kubernetes manifests, Helm charts, and CI/CD pipelines to automate data workflows.
   • Monitor and troubleshoot containerized applications to ensure high availability and reliability.

4. Performance Optimization:

   • Optimize data processing pipelines for performance and scalability.
   • Tune SQL queries and data storage configurations to handle large volumes of data efficiently.
   • Implement monitoring and logging solutions to track data pipeline performance and identify issues.

5. Collaboration and Communication:

   • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide support.
   • Collaborate with DevOps and infrastructure teams to integrate data solutions with existing systems.
   • Document data processes, workflows, and configurations for transparency and knowledge sharing.

Qualifications:

• Experience: Experience as a Data Engineer or in a similar role with a strong focus on Python, SQL, and cloud technologies.
• Technical Skills:
  • Proficiency in Python for data engineering tasks, including scripting and automation.
  • Advanced SQL skills for querying and manipulating data.
  • Hands-on experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and related data services.
  • Experience with containerization and orchestration technologies such as Docker and Kubernetes.

To Apply for this Job Click Here