The Senior Data Engineer, an essential role part of our Data Engineering team, will play a pivotal role in orchestrating and operationalizing the most-urgent data and analytics initiatives for our digital business initiatives. The Senior Data Engineer is accountable for the solutioning, building, and optimizing of business-critical data stores, data pipelines, and ensuring the testing and migration of these data stores/pipelines is done in a repeatable, continuous fashion. The Senior Data Engineer will also partner with fellow Data Engineers, IT Developers, and business leaders to ensure solution alignment with Data Quality and Management standards, encourage the adoption of new data management/storage/transformation technologies, and innovate new data automations and data sets that deliver efficiencies and insights to our businesses and clinicians
- Solution, Design, and Build ETL Pipelines
- Ability to work with both IT and business in integrating business requirements, analytics, logging, and monitoring into ETL pipelines
- Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management across large, complex datasets
- Incorporate modern data preparation, integration and AI-enabled metadata management tools and techniques where appropriate
- Demonstrate and Lead through modern data concepts, tooling, and practices
- Incorporate innovative and modern tools, techniques and architectures into the team’s solution design and development efforts
- Define and refine Data Mart/Data Lake architectures to ensure data sets continue to align to changing business needs and priorities
- Provide advice and guidance to leadership and team on best practices and guidelines for data analysis, warehousing, distribution, and pipeline management
- Actively and Effectively lead, participate in, and communicate in daily meetings, data model/pipeline reviews, script/algorithm reviews
- Ensure compliance and governance during data use
- Ensure that the data users and consumers use the data provisioned to them responsibly through data governance and compliance initiatives
- Participate in vetting and promoting content created in the business and by data analysts to the curated data catalog for governed reuse
- Help coordinate and establish standards for development
- Circulate and acquire buy-in from cross-organization team members about best practices
- Peer-review team work to ensure best design practices and evolve team development
- Review current standards and help implement new standards based on new principles and patterns
- Bachelor of Science in Computer Science, Business Administration or technology related field preferred.
- 5+ years or more of work experience in data management disciplines including data integration, modeling, optimization, and data quality, and/or other areas directly relevant to data engineering responsibilities and tasks.
- 5+ experience working in cross-functional teams and collaborating with business stakeholders in healthcare in support of a departmental and/or multi-departmental data management and analytics initiative.
- Strong Knowledge of basic programming concepts, data processing languages (e.g. Python, Scala, SQL), and systems implementation.
- Experience with processing, storaging, analyzing data from messaging systems like Kafka, RabbitMQ
- Fluent across varying business applications and corresponding datasets and storage/access mechanisms (PowerBI, Tableau, Oracle, SQL server, REST, SOAP, MongoDB)
- Ability to interface effectively with all levels and across organizational lines.
- Proficient in agile methodologies and capable of applying DevOps and increasingly DataOps (or Data DevOps) and Test Driven Development (TDD) principles to data pipelines to improve the communication, integration, reuse and automation of data flows between data managers and consumers across an organization
- Passionate about problem solving, process optimization, performance improvement, and standardization as it relates to data transformation, data movement, and data analysis
- The ability to manage delivery of multiple projects involving many vendors and staff members
- Strong communication skills to translate complex data solutions to executive leadership, staff, and vendors, and have regular discussions with executive business counterparts on data-related process optimizations
- Strong knowledge of Microsoft SQL Server and Azure resources, database, and data lake storage concepts
- Experience with large cloud based/third party datasets
- EHR System experience (e.g. Epic, Cerner, Meditech, AllScripts, NextGen)
- Experience with Healthcare data concepts, frameworks, and policies (HL7, FHIR, HIPAA)
- Experience in writing, understanding, and diagnosing intricate, optimized queries over large datasets