Job Description

Job title: Lead GCP Data Engineer

Job Location: Irving, TX / Atlanta GA / Baskin Ridge, NJ

Job Type : Contract

Job Description:

Mandatory Skills

• Design, build, and maintain scalable data pipelines using Cloud Dataflow, Apache Beam, Apache Spark, or BigQuery.
• Develop ETL/ELT workflows for data ingestion, transformation, and processing using Cloud Composer (Airflow), TIDAL, Dataform, or custom scripts.
• Optimize BigQuery performance through partitioning, clustering, and query tuning.
• Implement data governance, security, and compliance best practices within GCP.
• Work with Cloud Storage, Pub/Sub,Ni-Fi, Cloud SQL and Bigtable for real-time and batch data processing.
• Monitor and troubleshoot data pipeline performance, failures, and cost efficiency.
• Collaborate with data scientists, analysts, and software engineers to support business requirements.
• Ensure data quality, validation, and integrity using appropriate testing frameworks.
• Strong expertise in GCP services (BigQuery, Dataflow, Cloud Storage, Pub/Sub, Bigtable, Firestore, etc.).
• Proficiency in SQL, Python, and Java for data processing and automation.
• Experience with ETL/ELT workflows using Cloud Composer, Dataflow, or Dataform.
• Strong understanding of data modeling, warehousing, and distributed computing.
• Experience with real-time and batch processing architectures.
• Knowledge of CI/CD pipelines, Git, and DevOps best practices.
• Understanding of security and compliance standards (IAM, encryption, GDPR, HIPAA, etc.).

Preferred Qualifications:

• GCP certifications (e.g., Professional Data Engineer, Associate Cloud Engineer).
• Experience with machine learning pipelines on GCP (Vertex AI, AI Platform, etc.).
• Exposure to Kafka, Ni-Fi, or other streaming technologies.
• Experience with containerization and orchestration (Docker, Kubernetes, GKE).

Job Tags

Similar Jobs