About

I’m building an end-to-end Python pipeline to collect, decode, and store motorsport telemetry and session data from live JSON feeds. The goal is to have a robust, high-performance, and well-organized database that powers a dashboard for visualization and analysis. What I Have: -Example Python scripts that fetch and decode live JSON streams -Sample feeds from past sessions to validate parsing -Knowledge of the URLs and JSON structure Project Goals: 1.Python ETL pipeline that: - Continuously fetches live telemetry and session data - Decodes JSON packets and normalizes the data - Automatically inserts data into the databases 2. Database storage: - SQLite for season and session info (event/session metadata, results, laptimes, drivers, teams) - PostgreSQL for telemetry data (all drivers, all sessions, per lap and per channel) - Optimized schema for fast writes and reads, supporting queries for dashboard visualizations 3. Continuous ingestion: - The pipeline should run automatically and write data as it comes in - Freelancer must validate ingestion during 2 live event feeds to ensure data flows correctly 4. Access utilities: - Scripts or API to query the database easily for analysis - Examples for retrieving telemetry for specific drivers, sessions, or laps - Help files/documentation for using the ETL and query tools 5. Deployment: - Docker image or clear Linux deployment instructions so the system can run on a server reliably Requirements: -Python must be used for all scripts -Emphasis on robustness, clarity, performance, and well-structured code -Continuous ingestion must be proven with real or sample live feeds -Database choices are decided (but open for suggestions): SQLite (season/session) and PostgreSQL (telemetry) Deliverables: -Fully working Python ETL pipeline with continuous ingestion -Database schemas and indexes (clean and organized) -Access scripts or small API for querying data (season/session, telemetry) -Deployment instructions / Docker setup -Help files and example scripts demonstrating typical use cases -Validation showing ingestion works for at least 2 live events Acceptance Criteria: - Pipeline runs without manual intervention and ingests live data continuously - Queries on the telemetry database return the expected results in the expected format quickly - Scripts and documentation are clear and reproducible on a standard Linux server and locally. Budget is negotiable depending on experience, but full validation of all deliverables as described above is required.