US Patent 11836125 Scalable database dependency monitoring and visualization system

Computer systems and associated methods are disclosed to implement a database dependency monitoring system (DDMS) that periodically collects activity data from an inventory of databases and generates dependency graphs indicating data dependencies among the databases. In embodiments, the collection of the activity data may be implemented as three sets of jobs that are run periodically. In the first job set, the databases are checked to see if they can be connected to. In the second job set, queries are run in each of the available databases to generate output files containing their activity data. In a third job set, the output files are transformed and loaded into a metadata store, which can be used to generate the dependency graph based on the collected activity data. Advantageously, the disclosed system may be used to adaptively monitor a changing inventory of many databases, while causing little or no performance impact on the databases.