Phython fullstack

Job description

Strong expertise in SQL, PySpark, and Python to build scalable batch and distributed data pipelines. •Ability to write and tune complex SQL for analytics on distributed query engines. •Experience designing high-performance data processing using Spark SQL and DataFrames. •Proficiency in developing Python-based automation scripts and reusable data engineering components. •Hands-on experience with distributed SQL engines such as Starburst or similar technologies. •Practical knowledge of lakehouse architecture using Delta Lake, Iceberg, or equivalent platforms. •Experience working with cloud object storage such as S3, ADLS, or similar environments. •Understanding of ETL/ELT processes and managed data services for ingestion and transformation. •Solid knowledge of relational and non-relational databases as source systems. •Ability to implement data quality checks, including validation, monitoring, and reconciliation. •Experience in analytical data modeling, including dimensional and star schema design. •Familiarity with Airflow, Autosys, or similar orchestration tools. •Exposure to CI/CD, DevOps practices, and Git for collaborative pipeline development. •Strong skills in debugging data issues, validating transformations, and optimizing performance. •Proven problem-solving ability for complex data engineering challenges. •Ability to leverage AI coding tools like GitHub Copilot, Devin, or Claude to improve productivity and code quality. Interview Pattern (Strict): No repeating questions unless asked by the candidate. Partial answers are acceptable. No deep dive into techniques beyond whats listed. Focus on real-world experience, prompt safety, cloud setup, cost optimization, and HITL handling.