Hi everyone,
I recently completed the Data Engineering Professional Certificate program. I love how Joe Reis presents the knowledge in this program, starting with how to think as a data engineer in course one to fully understand the entire pipeline in the final course.
After this course, I want to discuss how practice goes further for junior-level data engineers like us. I recommend enrolling in the Data Engineer with AWS and Data Streaming program at Udacity. They also provided projects based on AWS, such as going through the NoSQL database with Apache Cassandra, going deeper into streaming data using Apache Kafka, and practicing the concept of a simple data warehouse and LakeHouse in AWS.
YouTube videos for other Apache family tools can help us learn by doing. For example, Dremio gives the concept and simple implementation of LakeHouse with Apache Iceberg on a local computer.
I am currently trying to do the projects from this program and the Udacity programs using my own AWS and Terraform to build the infrastructure from modules: VPCs, S3 buckets, Glue Jobs, and Redshift for simple data lake—LakeHouse architecture.
Understanding the Docker and Kubernetes concepts is helpful, as it can be seen as orchestration and DataOps in undercurrents. The last capstone project shows that Airflow, Superset, or DBT can be run as containers in an EC2 instance.
I am excited to start my career as a Data Engineer. I want to connect with other experienced data engineers who have taken this same or similar program to learn about their career paths and gain insights. Gaining knowledge means experiencing practical work or being closer to someone wiser than me. I’m particularly looking for a junior-level data engineer. I know the need for industrial experience in the data engineering field and hope to find opportunities to work with a senior engineer.
I see myself as having skills and tools on both ends of the data pipeline. My experience as an autonomous vehicle engineer involved working directly with the source systems, producing and managing the sensor data ( camera, lidar, and radar) that fuels data pipelines. Coupled with my academic background in the Master of Science in Computing and Data Science, I understand the downstream needs of data analysts and scientists, offering me a view to serve efficient and effective data solutions.
I am taking the CKAD certificate in two weeks and the AWS Solutions Architect Associate at the end of December, which I hope will equip me with skillful techniques to handle more problems in a working environment.
I graduated in January 2024 and am still actively looking for a job in the US. I’m also happy to share my resume and cover letter for feedback. I’ve provided my email address for this post. Any advice on tailoring my application materials to specific roles or companies would be greatly appreciated.
Thanks in advance for your help!
Personal email: noahtrth@gmail.com