Data Engineer - Remote
My IT LLC
Job ID: 4016554123
Location: San Francisco, CA
Description:
**Job Title:**
**Data Engineer
Duration: 6 Months
Location: SFO, CA (Remote)
End-client: Airbnb
Exp Level: 10 Years
Interview Process: 3 Rounds of Interview**
**Expertise:**
* 5-9+ years of relevant industry experience with a BS/Masters, or 2+ years with a PhD
* Experience with distributed processing technologies and frameworks, such as Hadoop, Spark, Kafka, and distributed storage systems (e.g., HDFS, S3)
* Demonstrated ability to analyze large data sets to identify gaps and inconsistencies, provide data insights, and advance effective product solutions
* Expertise with ETL schedulers such as Apache Airflow, Luigi, Oozie, AWS Glue or similar frameworks
* Solid understanding of data warehousing concepts and hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and columnar databases (e.g., Redshift, BigQuery, HBase, ClickHouse)
* Excellent written and verbal communication skills
**A Typical Day:**
* Design, build, and maintain robust and efficient data pipelines that collect, process, and store data from various sources, including user interactions, financial details, and external data feeds.
* Develop data models that enable the efficient analysis and manipulation of data for merchandising optimization. Ensure data quality, consistency, and accuracy.
* Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executor framework
* Collaborate with cross-functional teams, including Data Scientists, Product Managers, and Software Engineers, to define data requirements, and deliver data solutions that drive merchandising and sales improvements.
* Contribute to the broader Data Engineering community at Airbnb to influence tooling and standards to improve culture and productivity
* Improve code and data quality by leveraging and contributing to internal tools to automatically detect and mitigate issues.
* Skill Sets - Python, SQL (expert level), Spark and Scala (intermediate).
Remote: True
Min Salary: 0
Max Salary: 0
Salary Frequency: yearly
Posted Date:
URL: https://www.linkedin.com/jobs/view/4016554123
Craft Resume Craft Cover Letter