STAFF, DATA ENGINEER

Job description

About Team: Everyone has data, but the sheer volume of data at Walmart can be limitless. In the Data Engineering team, we help Walmart manage this data by building pipelines and data lakes to prepare big data for analysis and unlocking actionable insights in real-time. We also use cross-departmental data and machine learning to build a holistic view of true profitability, saving millions of dollars across item categories and geographies while assisting our leadership in making better decisions faster. What you'll do: Data Transformation and Integration: Extracts data from identified databases. Creates data pipelines and transform data to a structure that is relevant to the problem by selecting appropriate techniques. Develops knowledge of current analytics trends. Data Source Identification: Supports the understanding of the priority order of requirements and service level agreements. Helps identify the most suitable source for data that is fit for purpose. Performs initial data quality checks on extracted data. Data Modeling: Analyses complex data elements, systems, data flows, dependencies, and relationships to contribute to conceptual, physical, and logical data models. Develops the Logical Data Model and Physical Data Models including data warehouse and data mart designs. Defines relational tables, primary and foreign keys, and stored procedures to create a data model structure. Evaluates existing data models and physical databases for variances and discrepancies. Develops efficient data flows. Analyses data-related system integration challenges and proposes appropriate solutions. Code Development and Testing: Writes code to develop the required solution and application features by determining the appropriate programming language and leveraging business, technical and data requirements. Creates test cases to review and validate the proposed solution design. Creates proofs of concept. Tests the code using the appropriate testing approach. Deploys software to production servers. Contributes code documentation, maintains playbook, and provides timely progress updates. Applied Business Acumen: Provides recommendations to business stakeholders to solve complex business issues. Develops business cases for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Recommends new processes and ways of working. Participate in designing data application architecture with senior architects. Data Governance: Establishes, modifies, and documents data governance projects and recommendations. Implements data governance practices in partnership with business stakeholders and peers. Interprets company and regulatory policies on data. Educates others on data governance processes, practices, policies, and guidelines. Provides recommendations on needed updates or inputs into data governance policies, practices, or guidelines. Demonstrates up-to-date expertise and applies this to the development, execution, and improvement of action plans by providing expert advice and guidance to others. Supporting and aligning efforts to meet customer and business needs and building commitment for perspectives and rationales. Provides and supports the implementation of business solutions by building relationships and partnerships with key stakeholders. Identifying business needs, determining, and carrying out necessary processes and practices. Promotes and supports company policies, procedures, mission, values, and standards of ethics and integrity by training and providing direction to others in their use and application, ensuring compliance with them. Ensures business needs are being met by evaluating the ongoing effectiveness of current plans, programs, and initiatives. Applying suggestions for improving efficiency and cost effectiveness; and participating in and supporting community outreach events. Create and maintain optimal data pipeline architecture. Develop set processes for data modeling, mining and production. Create custom software components and analytics applications. Employ a variety of languages and tools to marry data systems together. Recommend ways to improve data reliability, efficiency, and quality. Work closely with all business units and engineering teams to develop strategy for long term data platform architecture. Design and implement data tools for analytics and data scientist team members to help them in building and optimizing in the Walmart ecosystem. Creates training documentation and trains end-users on data modeling. Oversees the tasks of less experienced programmers and stipulates system troubleshooting supports. What you'll bring: Well versed with Big Data Technologies, Spark, Cloud, Streaming, Kafka core, Kafka connect. Well versed with backend technologies and application development frameworks like Spring boot, React, Node.js. You have a proven track record coding with at least one programming language (e.g., Java, Scala, Python). You’re experienced in one of cloud computing platforms (e.g., GCP, Azure). You’re skilled in data modeling & data migration protocols. Experience with GCP (or any other similar cloud offerings), Data warehousing, BI preferred. Experience with the integration tools like Automic, Airflow. Experience in designing large scale distributed systems and well versed with end-to-end system designing. Experience in building highly scalable Big Data solutions and ETL ecosystems. Hands on knowledge in NoSQL like Druid, Cosmos DB etc. along with RDBMS like MySQL, Postgres is plus. Develop set processes for data mining, data modeling, and data production. -