Pyspark Jobs in Bangalore

Azure Cloud & Data ETL Developer

SACHA Engineering•

Bengaluru

•full time

Job Summary We are seeking a skilled and experienced Azure Cloud & Data ETL Developer to join our dynamic IT / Technology team. This is a hybrid role, offering a blend of remote work and in-office collaboration. The ideal candidate will possess a strong background in data engineering, cloud technologies, and ETL/ELT development, with a focus on the Azure ecosystem. Key Responsibilities The Azure Cloud & Data ETL Developer will be responsible for designing, developing, and maintaining robust and scalable data pipelines and solutions within the Azure cloud environment. This includes data ingestion, transformation, and loading processes, ensuring data quality, and adhering to data governance principles. The role also involves collaborating with cross-functional teams to understand data requirements and deliver effective solutions. Required Skills & Qualifications Azure Data Services: Demonstrated proficiency in utilizing various Azure data services, including Azure Data Factory (ADF) , Azure Databricks , and Azure Synapse Analytics , to build and manage data pipelines. ETL Development: Extensive experience in ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) development, including designing and implementing data integration solutions. Hybrid Data Architectures: A strong understanding of hybrid data architectures , enabling seamless data flow between on-premises and cloud environments. Cloud Migration: Experience in migrating data and applications to the Azure cloud platform. Data Engineering: Solid foundation in Data Engineering principles and practices. Data Modeling: Expertise in data modeling techniques, including the design and implementation of star schemas and snowflake schemas . Data Warehousing Concepts: A thorough understanding of data warehousing concepts and best practices. Programming Languages: Proficiency in Python and PySpark for data manipulation and processing. Data Ingestion & Integration: Experience with data ingestion and data integration processes, ensuring data accuracy and consistency. Data Quality & Governance: Knowledge of data quality and data governance principles and practices. DevOps & CI/CD: Experience with DevOps practices, including CI/CD (Continuous Integration/Continuous Delivery) pipelines, utilizing tools like Azure DevOps and GitHub Actions . Agile Methodology: Familiarity with Agile development methodologies. Performance Tuning & Indexing: Ability to perform performance tuning and indexing to optimize data query performance. Data Validation: Experience in implementing data validation processes to ensure data integrity. Azure Data Lake Storage (ADLS): Experience working with Azure Data Lake Storage (ADLS) . Communication Skills: Excellent communication skills , both written and verbal, to effectively collaborate with stakeholders. Problem-Solving Skills: Strong problem-solving skills to identify and resolve data-related issues. Gemini AI: Familiarity with Gemini AI and its applications in data processing. Experience Required: 6 - 10 years of relevant experience in data engineering and cloud development. This level of seniority requires a proven track record of designing and implementing complex data solutions, leading projects, and mentoring junior team members. What We Offer Salary: 18 LPA

Big query Architect

SACHA Engineering•

Bengaluru

•full time

Job Summary We are seeking an experienced Big Query Architect to join our team in a hybrid work environment. As a Big Query Architect, you will be responsible for designing and implementing large-scale data warehousing solutions using GCP BigQuery. With a strong background in data architecture and a passion for data-driven insights, you will play a critical role in shaping our data strategy and enabling business growth through data-driven decision-making. The ideal candidate will have a strong technical background, with a deep understanding of data warehousing principles, data modeling, and data governance. You will work closely with cross-functional teams to design and implement data pipelines, develop data products, and drive business outcomes through data-driven insights. Key Responsibilities Design and implement large-scale data warehousing solutions using GCP BigQuery, ensuring data quality, integrity, and performance. Develop and maintain complex data pipelines using Python and PySpark, ensuring seamless data integration and processing. Collaborate with stakeholders to understand business requirements and develop data-driven solutions to drive business outcomes. Ensure data governance and compliance with organizational data policies and procedures. Optimize data warehouse performance, scalability, and security. Required Skills & Qualifications The ideal candidate will possess a strong technical background in GCP BigQuery, Python, and PySpark. You will have a deep understanding of data warehousing principles, data modeling, and data governance. A bachelor's degree in Engineering ( B.E ) is required. Technical Skills: GCP BigQuery: Experience with designing and implementing large-scale data warehousing solutions. Python: Proficiency in developing data pipelines and data products. PySpark: Experience with developing and maintaining complex data pipelines. What We Offer The salary range for this position is up to 35,00,000. The successful candidate will have the opportunity to work in a hybrid environment, with a dynamic team of professionals. The role offers a unique blend of technical challenge and business impact, with the potential to drive significant business outcomes through data-driven insights.