← Back to projects

Operational reporting

Python Reporting & Data Collection Utilities

Crawler, API collectors, and Flask/PostgreSQL reporting tools

  • Python ·
  • Flask ·
  • PostgreSQL ·
  • Selenium ·
  • Cron jobs ·
  • Reporting
Summary

Quick overview

Built Python collectors and a Flask reporting dashboard that centralized operational data from multiple external platforms. The system used APIs, Selenium crawlers, cron jobs, user management, reporting, and bonus/malus calculations to replace manual checking and provide unified month-to-month comparisons.

Collection flowOperational reportingInterview topics

Details are anonymized and summarized without exposing internal code, customer data, or proprietary implementation details.

Project snapshot

Context, responsibility, outcome, and takeaway

Context

I built an operational reporting tool that collected performance and activity data from multiple external platforms. The goal was to replace manual checking across many sites with a centralized local dashboard where users could view unified statistics, compare current and previous periods, and detect large drops or changes in activity.

My role

I built Python-based data collectors, Selenium crawlers for sources without suitable APIs, API integrations where available, cron-based scheduled collection, local centralized storage, and a Flask dashboard for reporting, user management, bonus/malus calculations, and period comparisons.

Result

The tool reduced manual platform-by-platform checking and gave users a unified reporting view for operational performance, including online time, activity duration, revenue, bonus/malus calculations, and large month-to-month changes.

Takeaway

Operational automation is most useful when collection, scheduling, normalization, and reporting are separated enough to debug independently and trusted by non-technical users.

Operational reporting flow

Collectors, scheduling, dashboard, and business visibility

01

APIs where available

02

Selenium crawlers

03

Cron scheduling

04

Flask reports

Collection-to-reporting flow

How external platform data became centralized operational reporting

01

API / crawler

02

Cron job

03

Normalize

04

Local storage

05

Flask dashboard

Visual proof

Anonymized proof panels

Structured interface-style panels that show the project shape without exposing real users, client data, or private code.

Scheduled collection

Collector status

Platform A · API · Last run 08:00

Platform B · Selenium · Source constraints

Platform C · Cron · Waiting for window

  • APIs
  • Selenium
  • Cron jobs

Reporting

Monthly comparison

Current month activity · 142h

Previous month activity · 156h

Change detected · -9%

  • MoM change
  • Activity time
  • Revenue

Operations

Bonus / malus view

User A · bonus eligible

User B · requires review

User C · large drop flagged

  • User management
  • Rules
  • Flags
Implementation notes

Expandable details for the technical story

Data sources

Some sources exposed APIs, while others required crawling. The collection layer used direct API integrations where available and Python/Selenium crawlers for sources without suitable APIs, with source-specific session handling and isolated debugging boundaries.

Scheduling

Collectors ran through cron jobs at specific intervals. Different platforms had different collection windows, so jobs were scheduled according to the timing requirements of each source.

Problem solved

Before this tool, users had to manually check multiple sites to understand activity, online time, and revenue. The tool centralized the data locally and made it easier to compare performance across platforms and time periods.

Reporting value

The dashboard helped answer operational questions such as how the current month compared to the previous one, whether a profile/account had a major drop in activity, whether revenue decreased significantly, and which accounts needed attention.

Technical focus

Core responsibilities and engineering details

  • Used APIs where available and Selenium-based crawlers where no suitable API existed
  • Handled crawler-heavy sources with Python/Selenium while keeping collection logic isolated and debuggable
  • Scheduled platform-specific collectors with cron jobs at different intervals depending on source timing rules
  • Built a Flask dashboard with reporting screens, user management, bonus/malus calculation logic, and month-to-month comparison views
Additional technical details 3 more points
  • Centralized operational data locally so users could compare current and previous periods from one place
  • Exposed activity, online-time, revenue, and large period-over-period change visibility
  • Separated collection, normalization, storage, and reporting so each part could be debugged independently
What I can explain in an interview

Concrete topics this project supports

  • API collection vs Selenium crawling depending on external source capabilities
  • Cron-based scheduling for platform-specific collection windows
  • Crawler-heavy source handling with isolated collection, retry, and debugging boundaries
  • Flask reporting dashboard, user management, and bonus/malus calculation logic
  • Month-to-month comparison views for activity and revenue changes
  • Separating collection, normalization, storage, and reporting for easier debugging