The objective of the project was to automate KPI A-B validation and establish a robust data reconciliation framework. This entailed leveraging advanced data processing tools and frameworks to ensure accuracy and efficiency in financial reporting processes.

• Confronted recurring issues while automating KPI A-B IcedQ validation.
• The lack of accurate monitoring and validation mechanisms for aggregate column values caused discrepancies in their financial reporting.

• Designed and implemented an additional KPI A-B reconciliation rule using PySpark, Databricks, and IcedQ.
• Developed a robust one-to-one validation framework for aggregate column values.
• Integrated SQL and Azure Data Factory for seamless data ingestion and monitoring.
• Employed Databricks to process and analyze large datasets efficiently.