Azure Data Engineer
Location: Augusta, Georgia, Boston, Massachusetts, Concord, California, Florham Park, New Jersey, Hartford, Connecticut, Jersey City, New Jersey, New York, New York, Newark, New Jersey, Philadelphia, Pennsylvania, Providence, Rhode Island, Rochester, New York, Stamford, Connecticut, Syracuse, New York
Job Type: Full time
Data Engineering Senior Consultant
Do you enjoy making sure that information is accessible and easy to use? So do we.
You’re a data designer who knows how to find, store and present a range of information from different sources so that everyone can access what they need quickly and simply, and use it effectively.
You draw on your considerable experience in bringing data and statistics to life to solve sometimes complex problems, and you’re comfortable looking after several projects at once. You’re able to make your own decisions while at the same time supporting more junior team members.
About the job
Working in Data Engineering, you know the importance of data to business. You design and set up projects that bring together information from a variety of sources, to enable analysis and decision-making. You make sure that data is accessible and easy to use, so that it can be used for routine and ad-hoc analysis.
Day to day, you will:
Use your knowledge to plan and deliver data warehouse and storage
Take part in designing and running bespoke data services for individual projects
Stay up to date with business best practice in using and retrieving data
Design, develop, adapt and maintain data warehouse architecture and relational databases that support data mining
Customize storage and extraction, metadata, and information repositories
Create and use effective metrics and monitoring processes
Help to develop business intelligence tools
Support deal teams by providing subject knowledge and solutions for client proposals
Author reports that include key performance indicators, show where current operations can be improved, and identify the causes of any problems
Create and maintain report forms and formats, information dashboards, data generators and canned reports, as well as other information portals and resources
Travel as required.
You’re got great experience in data and analysis, and how to source, store and share information. You’re a problem solver who’s happy to work autonomously and to share their knowledge and skills, as well as guiding other team members.
Manage data pipelines: assist in creation of end-to-end data pipelines from external and internal sources to the data warehouse; create new pipelines, recreate config files as needed.Manage jobs: Create automation jobs (e.g. Azure Data Factory) to normalize and prepare data to be consumed by analytics and the business.Platform administration: Administration of existing in-house databases running in Azure, Make and implement recommendations for optimization and monitoring.Provide subject-matter expertise on best practices for data and platform management, and technology adoptionCollaboration: Work closely with data analysts to ensure the data model consistently feeds the desired end-user reports and queriesData governance: Experienced in developing and integrating software allowing for flexible and scalable data transformation with data quality controlsKnowledge of Python and PySparkAdvanced knowledge in entity and relationship extraction from unstructured dataUnderstanding of database table indexing, with emphasis on clustered columnstore tablesProficient in data handling, understanding, analysis and interpretation of resultsExperience with SQL technologies: Databricks (Spark), OR Azure SynapseData security: Must be able to demonstrate an understanding of data security at rest and in transitScale: Deliver performance at scale, and accurately anticipate system enhancement needsData integrity: Data manipulation, error identification and handling, and modeling
Compensation for roles at Avanade varies depending on a wide array of factors including but not limited to the specific office location, role, skill set and level of experience. As required by local law, Avanade provides a reasonable range of compensation for roles that may be hired in New York as set forth below and information on benefits offered is here.
Role Location: Range of Starting Pay for role
New York: $140,250 – $ 165,000