Pyspark Jobs

Machine Learning Engineer

SACHA Engineering•

Chennai

•Full-time

Machine Learning Engineer Job Description Job Summary We are seeking a skilled and motivated Machine Learning Engineer to join our growing team in Chennai. This role offers an exciting opportunity to contribute to cutting-edge projects within the IT / Technology industry. Key Responsibilities The Machine Learning Engineer will be responsible for designing, developing, and deploying machine learning models and pipelines. This includes collaborating with cross-functional teams to identify and solve complex business problems using data-driven solutions. Required Skills & Qualifications AI/ML : A strong understanding of artificial intelligence and machine learning principles, including various algorithms and techniques. Scikit-learn : Proficiency in utilizing Scikit-learn for model building, evaluation, and selection. PyTorch : Experience with PyTorch for deep learning model development and training. TensorFlow : Familiarity with TensorFlow for building and deploying large-scale machine learning models. LLM : Knowledge of Large Language Models (LLMs) and their applications. ETL : Experience with Extract, Transform, Load ( ETL ) processes for data preparation and integration. Big Query : Ability to work with Big Query for data warehousing and querying. Data Flow : Understanding of Data Flow concepts and their application in data processing. Python : Strong programming skills in Python for data analysis, machine learning, and scripting. Airflow : Experience with Airflow for orchestrating and scheduling data pipelines. PySpark : Proficiency in PySpark for distributed data processing and machine learning. SQL : Expertise in SQL for data querying and manipulation. GCS : Familiarity with Google Cloud Storage ( GCS ) for data storage and management. GitHub : Experience using GitHub for version control and collaboration. CI/CD : Knowledge of Continuous Integration and Continuous Deployment ( CI/CD ) practices. Dynatrace : Experience with Dynatrace for application performance monitoring. Analytical Skills : Exceptional analytical and problem-solving skills with the ability to interpret data and draw meaningful conclusions. Software Engineering : Solid foundation in software engineering principles and best practices. Design Practices : Understanding of software design patterns and principles. Documentation : Ability to create clear and concise technical documentation. DevOps : Familiarity with DevOps methodologies and tools. Pair Programming : Experience with Pair Programming techniques for collaborative development. TDD : Knowledge of Test-Driven Development ( TDD ) principles. Agile Ceremonies : Experience participating in Agile Ceremonies . PI Planning : Understanding of PI Planning within an Agile framework. Experience Required: 4 - 10 years of relevant experience. We are seeking a candidate with a proven track record of building and deploying machine learning solutions. This level of experience should demonstrate a strong understanding of the entire machine learning lifecycle, from data collection and preparation to model deployment and monitoring. What We Offer Salary:

Data Engineering Engineer 0

RLE India Private Limited•

Chennai

•Full-time

Data Engineering Engineer 0 Job Description Job Summary We are seeking a skilled Data Engineering Engineer to join our team in Chennai. This role offers an exciting opportunity to contribute to a dynamic environment within the IT / Technology industry. Key Responsibilities The specifics of the key responsibilities were not provided. This section will be updated as information becomes available. Required Skills & Qualifications The ideal candidate will possess a strong foundation in data engineering principles and a proven ability to leverage various technologies to build and maintain robust data pipelines. The following skills and qualifications are essential: Data Processing & Orchestration: Demonstrated proficiency in designing, building, and maintaining data pipelines using AIRFLOW and Airflow PySpark . Experience in orchestrating complex data workflows and ensuring data quality throughout the process. Cloud Platform Expertise: Extensive experience with Google Cloud Platform ( GCP ), including Dataproc , Big Query , Data Flow , and Data Fusion . Ability to leverage GCP services to build scalable and reliable data solutions. Infrastructure as Code: Solid understanding of infrastructure as code principles and experience using TERRAFORM to automate infrastructure provisioning and management. Containerization & CI/CD: Familiarity with containerization technologies and experience using Tekton for continuous integration and continuous delivery (CI/CD) of data engineering pipelines. Database Management: Proficiency in working with relational databases, including POSTGRES and Cloud SQL . Ability to design and optimize database schemas for performance and scalability. Programming Skills: Strong programming skills in Python , with the ability to write clean, efficient, and well-documented code. API Integration: Experience working with APIs for data ingestion and integration. AI/ML Awareness: A foundational understanding of AI/ML concepts and how data engineering supports AI/ML initiatives. Qualifications: A Bachelor's Degree is required. Experience Required: 7 - 7 years of relevant experience. This indicates a senior-level role requiring a proven track record of success in data engineering, with the ability to lead projects and mentor junior team members. What We Offer Salary: The salary information was not provided.

Data Engineering Engineer 2

RLE India Private Limited•

Chennai

•full time

Job Summary We are seeking a skilled Data Engineering Engineer 2 to join our growing team in Chennai. This role is critical to supporting our data-driven initiatives within the Technology industry. The position requires a strong foundation in data engineering principles and a passion for building robust and scalable data pipelines. Key Responsibilities This role will focus on the design, development, and maintenance of data pipelines and infrastructure. The successful candidate will collaborate with cross-functional teams to ensure data quality, reliability, and accessibility. Required Skills & Qualifications The ideal candidate will possess a strong understanding of data engineering concepts and a proven ability to implement them in a practical setting. The following skills and qualifications are essential: Google Cloud Platform (GCP) : Demonstrated proficiency in utilizing GCP services for data storage, processing, and analysis. BigQuery : Expertise in designing and implementing BigQuery solutions for large-scale data warehousing and querying. Data Flow : Experience with Data Flow for building and managing streaming data pipelines. Dataproc : Ability to leverage Dataproc for running and managing Apache Spark and Apache Hadoop clusters. Data Fusion : Familiarity with Data Fusion for building and managing data integration pipelines. TERRAFORM : Proficiency in using TERRAFORM for infrastructure as code and automating the deployment of data infrastructure. Tekton : Experience with Tekton for building and managing CI/CD pipelines for data engineering projects. Cloud SQL : Knowledge of Cloud SQL for managing relational databases in the cloud. AIRFLOW : Experience with AIRFLOW for orchestrating and scheduling complex data workflows. POSTGRES : Familiarity with POSTGRES database management. Airflow PySpark : Ability to utilize Airflow and PySpark for data processing and transformation. Python : Strong programming skills in Python for data manipulation, scripting, and automation. API : Understanding of API integration and data exchange. Big Data : Solid understanding of Big Data concepts and technologies. Apache Kafka : Experience with Apache Kafka for building and managing real-time data streaming applications. Bachelor's Degree : A Bachelor's Degree is required. Experience Required: 4 - 4 years of experience in a data engineering role. This level of experience indicates a seasoned professional capable of independently designing and implementing complex data solutions, as well as mentoring junior team members. What We Offer Salary: (No salary provided in the input data)

Data Engineering Engineer 2

RLE India Private Limited•

Chennai

•full time

Job Summary We are seeking a skilled Data Engineering Engineer 2 to join our growing IT / Technology team. This role will focus on building and maintaining robust data pipelines and infrastructure. Key Responsibilities This position will be responsible for the design, development, and maintenance of data solutions. The role requires a strong understanding of data engineering principles and a commitment to delivering high-quality, scalable solutions. Required Skills & Qualifications The ideal candidate will possess a strong foundation in data engineering practices and a proven ability to work with complex data systems. The following skills and qualifications are essential: Google Cloud Platform: Demonstrated proficiency in utilizing Google Cloud Platform services for data storage, processing, and analysis. BigQuery: Expertise in querying and manipulating large datasets using BigQuery . Data Flow: Experience designing and implementing data pipelines using Data Flow . Dataproc: Ability to leverage Dataproc for distributed data processing and analytics. Data Fusion: Knowledge of Data Fusion for building and managing data integration pipelines. TERRAFORM: Proficiency in infrastructure-as-code using TERRAFORM for automating infrastructure provisioning and management. Tekton: Experience with Tekton for building and managing CI/CD pipelines. Cloud SQL: Familiarity with Cloud SQL for managing relational databases in the cloud. AIRFLOW: Strong understanding of AIRFLOW for orchestrating complex data workflows. POSTGRES: Experience with POSTGRES database management. Airflow PySpark: Ability to utilize Airflow and PySpark for data processing and transformation. Python: Strong programming skills in Python for data manipulation and automation. APIs: Experience working with APIs for data integration and exchange. Big Data: A solid understanding of Big Data concepts and technologies. Apache Kafka: Experience with Apache Kafka for real-time data streaming. Education: A Bachelor's Degree is required. Experience: A minimum of 4 - 4 years of experience in a data engineering role is expected. This level of experience indicates a seasoned professional capable of working independently and contributing significantly to complex projects. What We Offer Salary:

Artificial Intelligence Senior Associate

RLE India Private Limited•

Chennai

•full time

Job Summary We are seeking a highly motivated and skilled Artificial Intelligence Senior Associate to join our growing team in India. This role is ideal for a candidate with a strong foundation in data science and a passion for leveraging AI/ML to solve complex business challenges within the Technology industry. Key Responsibilities The Artificial Intelligence Senior Associate will be responsible for a range of tasks related to the development, implementation, and maintenance of AI/ML solutions. This includes collaborating with cross-functional teams to identify opportunities for AI/ML applications, designing and building AI/ML models, and ensuring the scalability and reliability of these models in a production environment. The role requires a proactive approach to problem-solving and a commitment to continuous improvement. Required Skills & Qualifications AI/ML : A deep understanding of artificial intelligence and machine learning principles, including various algorithms and techniques for model building, training, and evaluation. Python : Proficiency in Python programming language for data manipulation, statistical analysis, and AI/ML model development. Google Cloud Platform (GCP) : Experience with GCP services, including Big Query , Dataproc , Data Fusion , and Cloud SQL , for data storage, processing, and analysis. Data/Analytics : Strong analytical skills with the ability to extract meaningful insights from large datasets and communicate findings effectively. Automation : Experience in automating data processing and AI/ML workflows to improve efficiency and reduce errors. Business Analysis : Ability to understand business requirements and translate them into technical specifications for AI/ML solutions. Business Transformation : A grasp of how AI/ML can be leveraged to drive business transformation and create competitive advantage. CI/CD : Familiarity with continuous integration and continuous delivery ( CI/CD ) practices for deploying and managing AI/ML models. ETL : Experience with extract, transform, load ( ETL ) processes for data integration and preparation. Jira : Proficiency in using Jira for project management and issue tracking. Data Flow : Understanding of data flow principles and techniques for designing efficient data pipelines. TERRAFORM : Experience with TERRAFORM for infrastructure as code. Tekton : Knowledge of Tekton for container-native CI/CD. AIRFLOW : Experience with AIRFLOW for orchestrating complex data workflows. Airflow PySpark : Ability to utilize Airflow with PySpark for large-scale data processing. POSTGRES : Familiarity with POSTGRES database. API : Understanding of API design and integration. GCP : Comprehensive knowledge of GCP ecosystem. Financial Reporting : Ability to apply AI/ML techniques to financial data and generate insightful reports. Leadership : Demonstrated leadership potential with the ability to guide and mentor junior team members. Strategic Communication : Excellent communication skills with the ability to present complex technical concepts to both technical and non-technical audiences. Communication Skills : Strong written and verbal communication skills. Analytical Skills : Exceptional analytical and problem-solving skills. Problem-Solving Skills : Ability to identify, analyze, and solve complex problems effectively. GitHub : Proficiency in using GitHub for version control and collaboration. Qualifications : A Bachelor's Degree and a Master's Degree are required. Experience Required : We are seeking candidates with 3-9 years of relevant experience in data science or a related field. This level of experience indicates a strong foundation in AI/ML principles and a proven ability to deliver impactful results. What We Offer Salary :

Machine Learning Engineering Engineer 2

RLE India Private Limited•

Chennai

•full time

Job Summary We are seeking a skilled and motivated Machine Learning Engineering Engineer 2 to join our growing team. This role is ideal for an individual with a strong foundation in machine learning principles and a passion for building and deploying innovative solutions within the IT / Technology industry. Key Responsibilities The responsibilities of this role will encompass a range of tasks related to the development, implementation, and maintenance of machine learning models and pipelines. The candidate will collaborate with cross-functional teams to ensure the successful integration of machine learning solutions into our existing infrastructure. Required Skills & Qualifications AI/ML: Demonstrated proficiency in Artificial Intelligence and Machine Learning concepts, including model selection, training, evaluation, and deployment. Scikit-learn: Experience utilizing Scikit-learn for various machine learning tasks, including data preprocessing, model selection, and evaluation. PyTorch: Familiarity with PyTorch , a widely used deep learning framework, for building and training complex neural networks. TensorFlow: Experience with TensorFlow , another popular deep learning framework, for developing and deploying machine learning models at scale. LLM: Understanding of Large Language Models ( LLM ) and their applications. ETL: Knowledge of ETL (Extract, Transform, Load) processes for data preparation and integration. Big Query: Experience with BigQuery for querying and analyzing large datasets. Data Flow: Familiarity with Data Flow for building and managing data processing pipelines. Python: Strong programming skills in Python , a versatile language widely used in machine learning. Airflow: Experience with Airflow for orchestrating and scheduling complex data workflows. PySpark: Proficiency in PySpark for distributed data processing and machine learning. SQL: Solid understanding of SQL for data querying and manipulation. GCS: Experience with GCS (Google Cloud Storage) for storing and managing data in the cloud. GitHub: Familiarity with GitHub for version control and collaborative development. CI/CD: Understanding of CI/CD (Continuous Integration/Continuous Deployment) principles and practices. Dynatrace: Experience with Dynatrace for application performance monitoring. Analytical Skills: Strong analytical and problem-solving skills, with the ability to interpret data and draw meaningful conclusions. Software Engineering: Solid foundation in Software Engineering principles and best practices. Design Practices: Knowledge of software design patterns and principles. Documentation: Ability to create clear and concise technical documentation. DevOps: Understanding of DevOps methodologies and practices. Pair Programming: Experience with Pair Programming techniques for collaborative code development. TDD: Familiarity with TDD (Test-Driven Development) methodologies. Agile: Experience working within an Agile development environment. Education: Bachelor's Degree required. Experience: 1 - 4 years of relevant experience. This level is expected to be a mid-level contributor, capable of working independently and contributing to team goals. What We Offer Salary:

Azure Cloud & Data ETL Developer

SACHA Engineering•

Bengaluru

•full time

Job Summary We are seeking a skilled and experienced Azure Cloud & Data ETL Developer to join our dynamic IT / Technology team. This is a hybrid role, offering a blend of remote work and in-office collaboration. The ideal candidate will possess a strong background in data engineering, cloud technologies, and ETL/ELT development, with a focus on the Azure ecosystem. Key Responsibilities The Azure Cloud & Data ETL Developer will be responsible for designing, developing, and maintaining robust and scalable data pipelines and solutions within the Azure cloud environment. This includes data ingestion, transformation, and loading processes, ensuring data quality, and adhering to data governance principles. The role also involves collaborating with cross-functional teams to understand data requirements and deliver effective solutions. Required Skills & Qualifications Azure Data Services: Demonstrated proficiency in utilizing various Azure data services, including Azure Data Factory (ADF) , Azure Databricks , and Azure Synapse Analytics , to build and manage data pipelines. ETL Development: Extensive experience in ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) development, including designing and implementing data integration solutions. Hybrid Data Architectures: A strong understanding of hybrid data architectures , enabling seamless data flow between on-premises and cloud environments. Cloud Migration: Experience in migrating data and applications to the Azure cloud platform. Data Engineering: Solid foundation in Data Engineering principles and practices. Data Modeling: Expertise in data modeling techniques, including the design and implementation of star schemas and snowflake schemas . Data Warehousing Concepts: A thorough understanding of data warehousing concepts and best practices. Programming Languages: Proficiency in Python and PySpark for data manipulation and processing. Data Ingestion & Integration: Experience with data ingestion and data integration processes, ensuring data accuracy and consistency. Data Quality & Governance: Knowledge of data quality and data governance principles and practices. DevOps & CI/CD: Experience with DevOps practices, including CI/CD (Continuous Integration/Continuous Delivery) pipelines, utilizing tools like Azure DevOps and GitHub Actions . Agile Methodology: Familiarity with Agile development methodologies. Performance Tuning & Indexing: Ability to perform performance tuning and indexing to optimize data query performance. Data Validation: Experience in implementing data validation processes to ensure data integrity. Azure Data Lake Storage (ADLS): Experience working with Azure Data Lake Storage (ADLS) . Communication Skills: Excellent communication skills , both written and verbal, to effectively collaborate with stakeholders. Problem-Solving Skills: Strong problem-solving skills to identify and resolve data-related issues. Gemini AI: Familiarity with Gemini AI and its applications in data processing. Experience Required: 6 - 10 years of relevant experience in data engineering and cloud development. This level of seniority requires a proven track record of designing and implementing complex data solutions, leading projects, and mentoring junior team members. What We Offer Salary: 18 LPA

Big query Architect

SACHA Engineering•

Bengaluru

•full time

Job Summary We are seeking an experienced Big Query Architect to join our team in a hybrid work environment. As a Big Query Architect, you will be responsible for designing and implementing large-scale data warehousing solutions using GCP BigQuery. With a strong background in data architecture and a passion for data-driven insights, you will play a critical role in shaping our data strategy and enabling business growth through data-driven decision-making. The ideal candidate will have a strong technical background, with a deep understanding of data warehousing principles, data modeling, and data governance. You will work closely with cross-functional teams to design and implement data pipelines, develop data products, and drive business outcomes through data-driven insights. Key Responsibilities Design and implement large-scale data warehousing solutions using GCP BigQuery, ensuring data quality, integrity, and performance. Develop and maintain complex data pipelines using Python and PySpark, ensuring seamless data integration and processing. Collaborate with stakeholders to understand business requirements and develop data-driven solutions to drive business outcomes. Ensure data governance and compliance with organizational data policies and procedures. Optimize data warehouse performance, scalability, and security. Required Skills & Qualifications The ideal candidate will possess a strong technical background in GCP BigQuery, Python, and PySpark. You will have a deep understanding of data warehousing principles, data modeling, and data governance. A bachelor's degree in Engineering ( B.E ) is required. Technical Skills: GCP BigQuery: Experience with designing and implementing large-scale data warehousing solutions. Python: Proficiency in developing data pipelines and data products. PySpark: Experience with developing and maintaining complex data pipelines. What We Offer The salary range for this position is up to 35,00,000. The successful candidate will have the opportunity to work in a hybrid environment, with a dynamic team of professionals. The role offers a unique blend of technical challenge and business impact, with the potential to drive significant business outcomes through data-driven insights.