The 21st Minute


Harnessing the Speed of SQL

160 Million Records Daily

When tasked with creating a customized fuzzy-logic algorithm for duplicate invoice detection that could be used on big data, Kurt Johnson and Ricardo Murillo of Audimation Services Solutions Development Team were at a loss. They needed a solution that could not only handle processing roughly 160 million records at a time, but could also do it on a daily basis in a practical amount of time.

They looked at several applications that included specific analytics for duplicate testing but none of them fit the needs of the situation. That’s when they decided to try SQL (Structured Query Language), the language used by database professionals worldwide. They developed their SQL-based fuzzy duplicate invoice algorithms which match on variations of fields including vendor, date, amount, and invoice number. Their customized SQL algorithms performed successfully and at a speed that makes handling massive data sets practical.

Kurt and Ricardo believe their customized algorithms have great potential when deployed in continuous monitoring environments, such as CaseWare Monitor and hope to expand their usage into other analytics like address matching. Their goal is to achieve quality of output for fuzzy-matching in reasonable time frames no matter the size of the dataset. Their creativity and skills are making this goal a reality.

Automate Procedures , News

Posted By


Related Posts
Task Automation Using IDEA
Apr 16 Audit scenarios rarely require an entirely unique process. Having a preferred set of tests ready to go is a great time saver, but that can be further improved i...
Gain Productivity with RPA
Apr 16 Leaders of professional associations are urging auditors to embrace emerging technologies to advance the industry, create audit efficiencies and manage organiza...
5 Ways Manufacturers are Using Analytics to Make Data-Driven Decisions
Nov 20 At times, any manufacturing enterprise can feel like a web of mysteries and riddles. Just keeping track of all the moving parts—from supplier relationships to...

This website has been designed for modern browsers. Please update. Update my browser now