Data Engineer - Lead
10+years
Introduction
We are looking for 10+years experienced candidates for this role.
Responsibilities include:
- Develop and maintain scalable ETL data pipelines using GCP services.
Integrate structured and unstructured data into centralized data platforms. - Build data pipelines leveraging BigQuery, Cloud Storage, Dataflow, and Pub/Sub.
- Collaborate with data scientists and analysts to ensure data availability for analytics.
- Manage and optimize data warehouse environments, particularly BigQuery.
- Design and implement real-time data streaming using Cloud Pub/Sub and Dataflow.
- Implement data governance and ensure high-quality data standards.
Use Cloud Functions and Cloud Composer for task automation and orchestration. - Develop data transformation scripts using Python, Java, or Scala.
- Optimize database performance for cost and scalability.
- Work with cross-functional teams to meet business data needs.
Participate in code reviews, troubleshooting, and applying best practices. - Use Apache Airflow or Cloud Composer for pipeline orchestration.
Implement Terraform or similar IaC tools for managing cloud infrastructure. - Use SQL to query and manage large datasets in BigQuery.
- Leverage Docker and Kubernetes for containerized environments.
Collaborate with DevOps teams to build CI/CD pipelines for data workflows. - Communicate complex technical concepts to non-technical stakeholders.
- Identify and resolve performance bottlenecks in data pipelines.
- Stay updated with emerging GCP technologies and data engineering best practices.
Primary Skills :
- 7+ years of data engineering experience, 4+ years with GCP data services.
- Proficiency with BigQuery, Cloud Storage, Dataflow, Pub/Sub.
- Experience with Python, Java, or Scala for data processing.
- Familiarity with Apache Airflow or Cloud Composer.
- Strong experience with SQL and data warehousing.
- Google Cloud certification preferred.
- Good communication and teamwork skills.
Job Details
Role:
Data Engineer - Lead
Location :
Trivandrum/Bangalore
Close Date :
28-03-2025
Interested candidates may forward their detailed resumes to Careers@reflectionsinfos.com along with their notice period, current and expected CTC details. This is to notify jobseekers that some fraudsters are promising jobs with Reflections Info Systems for a fee. Please note that no payment is ever sought for jobs in Reflections. We contact our candidates only through our official website or LinkedIn and all employment related mails are sent through the official HR email id. Please contact careers@reflectionsinfos.com for any clarification/ alerts on this subject.
Apply Now
Recent Jobs