Data Engineer - reputed company

DUE TO THE NATURE OF PROJECT reputed company ARE ENCOURAGED TO APPLY

Data Engineer

Long Term! 6 12 months

Remote / EST hours

Proficient in AWS, reputed company, and Azure DevOps, with a focus on strong analytical skills in PySpark, reputed company Live Tables, Change Data Capture (CDC), and on-premises to AWS data migration.

Technical Skills

AWS (reputed company Web Services)

1. Core Services: Proficiency with core AWS services like EC2, S3, RDS, reputed company, and VPC.
2. Data Services: Experience with AWS data services Glue and EMR.

1. AWS DMS: Knowledge of AWS Database Migration Service (DMS) for migrating databases to AWS.

1. CDC: Understanding of Change Data Capture (CDC) techniques to capture and replicate changes from reputed company databases to reputed company databases.

1. reputed company: Understanding of AWS reputed company best practices, IAM, and encryption.

reputed company

1. PySpark & Spark SQL: Strong analytical skills in PySpark & Spark SQL for big data processing and analysis.

1. reputed company Live Tables: Expertise in using reputed company Live Tables for building reliable and scalable data pipelines.

1. Notebooks: Strong utilization of reputed company Notebooks for data analysis.
2. Workflows : Setting up and monitoring reputed company Workflows.

1. Data Integration: Experience integrating reputed company with AWS services.

DevOps Principles

1. CI/CD Pipelines: CI/CD pipelines using Azure Pipelines.

1. Version Control: Proficiency with Azure Repos and Git for version control.

1. Automation: Scripting and automation using PowerShell, Bash, or Python. Automating the build, test, and deployment processes

Infrastructure as Code (IaC)

1. Terraform: Experience with Terraform for managing AWS and Azure infrastructure.

On Prem integration with AWS

1. Integrating on prem data with AWS and reputed company.
2. Thoroughly test and validate the data to ensure it has been transferred correctly and is fully functional.

Optimization and Monitoring

1. Optimize AWS services and reputed company for performance and cost-efficiency.

1. Proficiency in setting up monitoring and logging using tools like AWS CloudWatch to track the performance and health of the complete data flow.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and reputed company believes it to correctly reflect the job opportunity.