Senior Data Engineer-Databricks
Organization: Versant
Location: New York, New York, United States
Company DescriptionVERSANT is a leading force in news, sports and entertainment - home to iconic and trusted brands that inspire, inform, and delight audiences. Our unique combination of content, technology and services enriches the cultural fabric, igniting passions, sparking conversations, and connecting people to what they love most.As an independent, publicly traded company, VERSANT brings together powerhouse cable networks - including USA Network, CNBC, MS NOW (formerly MSNBC), Oxygen, E!, SYFY, and Golf Channel - with dynamic digital and direct-to-consumer brands such as Fandango, Rotten Tomatoes, GolfNow, GolfPass, and SportsEngine. Together, these businesses reflect our commitment to delivering exceptional experiences across every screen and service.VERSANT is an industry-changing media company fueled by innovation and an entrepreneurial spirit. With a strong foundation and a forward-looking vision, VERSANT empowers creativity, embraces change, and drives connection in an ever-evolving world.Job DescriptionThe Data Engineering team is seeking a Senior Data Engineer to help design, build, and scale the modern data platform that powers analytics, data science, and data products across Versant brands. In this role, you’ll collaborate closely with Data Product, Data Science, Analytics, and Engineering teams to deliver reliable, high-impact data solutions used by hundreds of internal users. We're looking for engineers who enjoy hands-on development, take ownership of production systems, and influence implementation through collaboration and technical expertise.Key ResponsibilitiesDesign and implement lakehouse architecture using Delta Lake, including medallion pipeline patterns (Bronze/Silver/Gold), schema enforcement, and time travelBuild and operate batch and real-time ingestion pipelines leveraging Databricks Auto Loader, Structured Streaming, and Change Data Capture patternsImplement data governance and security using Unity Catalog, RBAC, and compliance-driven practices for sensitive environmentsOptimize performance and manage costs through FinOps strategies, including cluster sizing, workload tagging, Spark tuning, and Photon accelerationDesign, implement, and maintain CI/CD pipelines and orchestration workflows using Databricks Workflows, Delta Live Tables, and tools such as AirflowCollaborate with Data Science teams on ML workflows, including MLflow, feature store integration, and model lifecycle managementEnsure data quality, observability, and lineage across media-specific datasets such as streaming logs, ad impressions, and audience metricsProvide technical mentorship through code reviews, pairing and knowledge sharingQualificationsBachelor’s degree in Computer Science, Data Engineering, or equivalent practical experience5+ years of experience building production data solutions in cloud environments (e.g., Databricks, AWS)Expertise in SQL and Python, with strong proficiency in Spark-based data processingHands-on experience building batch or streaming production data pipelines using distributed processing frameworks (e.g., Spark, Flink) and query engines such as PrestoProficiency with orchestration tools such as Apache Airflow or Dagster, and experience designing systems with monitoring, logging, and alerting for proactive issue detectionExperience working with modern data architectures, including event-driven and distributed systemsProficiency with Git and collaborative development workflowsSolid understanding of infrastructure, networking, and data security fundamentalsPreferred QualificationsExperience building Lakehouse platforms and medallion pipelines in DatabricksFamiliarity with Unity Catalog, data governance, and compliance frameworks (e.g., PCI)Hands-on experience with CI/CD pipelines, orchestration tools, and infrastructure-as-codeExperience with Lakeflow Spark Declarative Pipelines (SDP), MLflow, feature stores, and MLOps practicesBackground in media and entertainment data (e.g., video metadata, ad tech, audience analytics)Experience building data platforms within the media industry, with a strong understanding of audience analytics. Hands-on experience with audience research tools and methodologies (e.g., Nielsen), clickstream data processing, and identity graph or identity spine solutions for identity resolution and personalizationComfortable using AI-assisted development tools (e.g., ChatGPT)Databricks or cloud certifications (e.g., Databricks Certified Data Engineer)What Sets You Apart Passion for clean, reliable, and scalable data systemsStrong communication and collaboration skillsAbility to balance near-term delivery with thoughtful technical designCuriosity and a growth mindsetA team-first attitude who supports and uplifts othersAdditional InformationAs part of our selection process, candidates may be required to attend an in-person interview with a VERSANT employee in New York. VERSANT's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law.If you are a qualified individual with a disability or a disabled veteran and require support throughout the application and/or recruitment process as a result of your disability, you have the right to request a reasonable accommodation. You can submit your request to [email protected].VERSANT is committed to fair and equitable compensation practices. We include a good faith pay range for each position to comply with applicable state and local pay transparency laws and to promote equity across our organization. Actual compensation will be based on factors such as the candidate's skills, qualifications, experience, and location and may include additional forms of compensation and benefits such as health insurance, retirement plans, paid time off, etc.Salary range: $140,000 - $160,000Additional InformationAs part of our selection process, external candidates may be required to attend an in-person interview with a VERSANT Media employee at one of our locations prior to a hiring decision. VERSANT Media's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law.If you are a qualified individual with a disability or a disabled veteran and require support throughout the application and/or recruitment process as a result of your disability, you have the right to request a reasonable accommodation. You can submit your request to [email protected].VERSANT Media is committed to fair and equitable compensation practices. We include a good faith pay range for each position to comply with applicable state and local pay transparency laws and to promote equity across our organization. Actual compensation will be based on factors such as the candidate's skills, qualifications, experience, and location and may include additional forms of compensation and benefits such as health insurance, retirement plans, paid time off, etc.VERSANT Media is not accepting unsolicited assistance from search firms for this employment opportunity. All resumes submitted by search firms to any employee at VERSANT via-email, the Internet, or in any form and/or method without a valid written Statement of Work in place for this position from VERSANT's Talent Acquisition team will be deemed the sole property of VERSANT. No fee will be paid in the event the candidate is hired by VERSANT as a result of the referral or through other means.Business Segment: Versant Revenue & Digital Compensation: USD140000 - USD160000 - yearlyApply: https://jobs.smartrecruiters.com/Versant3/744000107402005-senior-data-engineer-databricks