Key Responsibilities
Design, develop, and maintain scalable ETL pipelines using PySpark and Python to process large-scale datasets across distributed environments.
Implement complex data transformation logic and optimize Spark jobs for performance and cost-efficiency.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
Integrate data from multiple sources (structured, semi-structured, unstructured) into unified data models.
Ensure data quality, consistency, and governance through validation, monitoring, and error handling mechanisms.
Participate in system design discussions and contribute to architectural decisions for big data platforms.
Document technical solutions and maintain version control using Git and CI/CD tools like Jenkins or Azure DevOps.
Required Skills
* Strong programming skills in Python and PySpark.
* Hands-on experience with Apache Spark, Hadoop, Hive, and HDFS.
* Proficiency in SQL and working with both relational and NoSQL databases.
* Strong debugging, performance tuning, and problem-solving skills.
...how much is at stake for those at-risk or affected by HIV, and they show up every day 100% committed to the patients. The Medical Science Liaison (MSL) role is a field-based, customer-facing, non-promotional medical and scientific position within the NYC area. The MSL...
...Technical Services Team. As an Entertainment Technician you will be responsible for running assigned tracks for stage productions and special events, assist with load in and load out of show product, assist with parade duties and assist with other technical services...
...AFIMAC is seeking licensed Security Guards for upcoming projects throughout the province of Alberta ***Please note this is a temporary contract position, so it may not be suitable to someone currently working full time***~$20.00 per hour before overtime ~ Hotel accommodations...
...accomplished Industry Managing Partner to lead the growth of our Consulting & Engineering Services (Applications, Data & AI) within the High-Tech sector in USA. The ideal candidate will bring deep domain expertise, strong client advisory skills, and a proven ability to drive...
....00 AnnuallyLocation : Klamath Falls, ORJob Type: Full-TimeJob Number: 202200098Department: PoliceDivision: POLICEOpening Date: 07/28/2025SummaryThe Police Records Specialist is an administrative position. Distinguishing career features of this position include...