The 21st Minute


Harnessing the Speed of SQL

160 Million Records Daily

When tasked with creating a customized fuzzy-logic algorithm for duplicate invoice detection that could be used on big data, Kurt Johnson and Ricardo Murillo of Audimation Services Solutions Development Team were at a loss. They needed a solution that could not only handle processing roughly 160 million records at a time, but could also do it on a daily basis in a practical amount of time.

They looked at several applications that included specific analytics for duplicate testing but none of them fit the needs of the situation. That’s when they decided to try SQL (Structured Query Language), the language used by database professionals worldwide. They developed their SQL-based fuzzy duplicate invoice algorithms which match on variations of fields including vendor, date, amount, and invoice number. Their customized SQL algorithms performed successfully and at a speed that makes handling massive data sets practical.

Kurt and Ricardo believe their customized algorithms have great potential when deployed in continuous monitoring environments, such as CaseWare Monitor and hope to expand their usage into other analytics like address matching. Their goal is to achieve quality of output for fuzzy-matching in reasonable time frames no matter the size of the dataset. Their creativity and skills are making this goal a reality.

Automate Procedures , News

Posted By


Related Posts
5 Ways Manufacturers are Using Analytics to Make Data-Driven Decisions
Nov 20 At times, any manufacturing enterprise can feel like a web of mysteries and riddles. Just keeping track of all the moving parts—from supplier relationships to...
Conflicts of Interest
Nov 15 Conflicts of interest (COI) occur when an employee must choose between the financial interests of the company and their own personal benefit. Over the years, I&...
Fraud Week Edition: Ghosts are Real! Ghost Employee & Vendor
Nov 14 Ghost Employees Ghost employees and payroll fraud can take different forms including fictitious employees, employees who do not work, and continuous payment af...

This website has been designed for modern browsers. Please update. Update my browser now