Splunk Inc.
Record expansion and reduction based on a processing task in a data intake and query system

Abstract:

Systems and methods are described for processing records associated with a query that identifies an association between two data fields. The system can obtain a chunk of data that includes multiple records based on a query received by a data intake and query system. At least one record can include multiple sub-records that share a field value for at least one field. The system can generate a record from each sub-record and assign the generated records to one or more groups of partitions. The system can combine record data of generated records assigned to one partition of a group of partitions and then combine record data across the group of partitions. The system can process the results of the combination of records across the group of partitions based on the query.