Brian Bowman

An apparatus includes a processor to: instantiate data buffers of a queue, reading threads, and provision threads; within each reading thread, use an identifier provided in a data buffer of the queue to retrieve the corresponding data set part and part metadata from storage device(s), and store both within the data buffer; operate the queue as a (FIFO) buffer; within each provision thread, retrieve a row group from among multiple row groups and corresponding metadata from within the data… Show more

An apparatus includes a processor to: instantiate data buffers of a queue, reading threads, and provision threads; within each reading thread, use an identifier provided in a data buffer of the queue to retrieve the corresponding data set part and part metadata from storage device(s), and store both within the data buffer; operate the queue as a (FIFO) buffer; within each provision thread, retrieve a row group from among multiple row groups and corresponding metadata from within the data buffer, use information in the metadata to decompress at least one column, and provide the data values of the row group to the requesting device or an application routine; and in response to each instance of storage of a data set part within a data buffer of the queue, analyze the availability of storage space and/or of processing resources to determine whether to dynamically adjust the quantity of reading threads. Show less

An apparatus includes a processor to: within each reading thread, retrieve a data set part and corresponding part metadata from storage device(s), analyze row group metadata for each row group within the data set part to identify candidate row group(s) meeting specified criteria, and store the candidate row group(s) and corresponding row group metadata within a data buffer of a queue; operate the queue as a FIFO buffer; within each provision thread, retrieve one of multiple row groups and… Show more

An apparatus includes a processor to: within each reading thread, retrieve a data set part and corresponding part metadata from storage device(s), analyze row group metadata for each row group within the data set part to identify candidate row group(s) meeting specified criteria, and store the candidate row group(s) and corresponding row group metadata within a data buffer of a queue; operate the queue as a FIFO buffer; within each provision thread, retrieve one of multiple row groups and corresponding metadata from within the data buffer, use information in the metadata to identify rows meeting the criteria, and provide those rows to the requesting device or an application; and in response to each instance of storage of a data set part within a data buffer of the queue, analyze the availability of storage space and/or of processing resources to determine whether to dynamically adjust the quantity of reading threads. Show less

An apparatus includes a processor to: within each collection thread, assemble a row group from stored rows, generate row group metadata corresponding to the row group, and store the row group and row group metadata within a data buffer of a queue; operate the queue as a FIFO buffer; within each aggregation thread, retrieve multiple row groups and corresponding row group metadata from multiple data buffers of the queue, assemble a data set part from the multiple row groups, generate part… Show more

An apparatus includes a processor to: within each collection thread, assemble a row group from stored rows, generate row group metadata corresponding to the row group, and store the row group and row group metadata within a data buffer of a queue; operate the queue as a FIFO buffer; within each aggregation thread, retrieve multiple row groups and corresponding row group metadata from multiple data buffers of the queue, assemble a data set part from the multiple row groups, generate part metadata that includes the row group metadata, and transmit, to storage device(s) and/or a requesting device, the data set part and/or the part metadata; and in response to each retrieval of at least a row group from a data buffer of the queue for an aggregation thread, analyze availability of storage space within the node device to determine whether to dynamically adjust the quantity of data buffers. Show less

An apparatus includes a processor to: instantiate collection threads, data buffers of a queue, and aggregation threads; within each collection thread, assemble a row group from a subset of the multiple rows, reorganize the data values row-wise to columnar organization, and store the row group within a data buffer of the queue; operate the buffer queue as a FIFO buffer; within each aggregation thread, retrieve multiple row groups from multiple data buffers of the queue, assemble a data set part… Show more

An apparatus includes a processor to: instantiate collection threads, data buffers of a queue, and aggregation threads; within each collection thread, assemble a row group from a subset of the multiple rows, reorganize the data values row-wise to columnar organization, and store the row group within a data buffer of the queue; operate the buffer queue as a FIFO buffer; within each aggregation thread, retrieve multiple row groups from multiple data buffers of the queue, assemble a data set part from the multiple row groups, transmit, to storage device(s) via a network, the data set part; and in response to each instance of retrieval of a row group from a data buffer of the buffer queue for use within an aggregation thread, analyze a level of availability of at least storage space within the node device to determine whether to dynamically adjust the quantity of data buffers of the buffer queue. Show less

An apparatus includes a processor component to: transmit node device identifiers to multiple node devices to define an ordering thereamong; following block exchanges redistributing the subsets among a reduced number of node devices, receive sizes of blocks or sub-blocks of data within each subset from the reduced number of node devices; based on the received sizes, generate map data organized to define an ordering among the blocks stemming from the ordering among the multiple node devices;… Show more

An apparatus includes a processor component to: transmit node device identifiers to multiple node devices to define an ordering thereamong; following block exchanges redistributing the subsets among a reduced number of node devices, receive sizes of blocks or sub-blocks of data within each subset from the reduced number of node devices; based on the received sizes, generate map data organized to define an ordering among the blocks stemming from the ordering among the multiple node devices; determine whether the total size of the map data and metadata, together, exceeds a minimum size for data transmissions to storage device(s); and in response to the total size exceeding the minimum size, form the map data and metadata into segment(s) that each fit the minimum size and a maximum size, and transmit the segment(s) at least partially in parallel with other segments of the blocks transmitted by the reduced number of node devices. Show less

An apparatus includes a processor component of a first node device caused to receive data block encryption data and an indication of size of an encrypted data block distributed to the first node device for decryption, and in response to the data set being of encrypted data: receive an indication of the quantity of sub-blocks within the encrypted data block, and a hashed identifier for each data sub-block; use the data block encryption data to decrypt the encrypted data block to regenerate data… Show more

An apparatus includes a processor component of a first node device caused to receive data block encryption data and an indication of size of an encrypted data block distributed to the first node device for decryption, and in response to the data set being of encrypted data: receive an indication of the quantity of sub-blocks within the encrypted data block, and a hashed identifier for each data sub-block; use the data block encryption data to decrypt the encrypted data block to regenerate data set portions from the data sub-blocks; analyze the hashed identifier of each data sub-block to determine whether all data set portions are distributed to the first node device for processing; and in response to a determination that at least one data set portion is to be distributed to a second node device for processing, transmit the at least one data set portion to the second node device.
Show less

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meeting the search criteria, and in response to a determination that the data cell includes such a data… Show more

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meeting the search criteria, and in response to a determination that the data cell includes such a data record, use a unique values index in the cell index to search the data records of the data cell to identify one or more data records meeting the search criteria; and in response to identifying at least one data record meeting the search criteria, provide an indication that at least the data cell includes at least one data record meeting the search criteria. Show less

Abstract
An apparatus includes a processor component to receive a node device identifier defining an ordering among multiple node devices and among multiple blocks of data distributed among the multiple node devices, and transmit a size of a first subset of the multiple blocks stored within the node device to a control device. In response to receiving instructions to receive a second subset from another node device, perform operations including: receive and store the second subset; group the… Show more

Abstract
An apparatus includes a processor component to receive a node device identifier defining an ordering among multiple node devices and among multiple blocks of data distributed among the multiple node devices, and transmit a size of a first subset of the multiple blocks stored within the node device to a control device. In response to receiving instructions to receive a second subset from another node device, perform operations including: receive and store the second subset; group the blocks of data of the first and second subsets into multiple segments in an order that corresponds to the ordering among the multiple blocks, wherein each segment is sized to fit minimum and maximum sizes for transmission to storage device(s); transmit the multiple segments to the storage device(s); and relay multiple segment identifiers from the storage device(s) to the control device in an order corresponding to the ordering among the multiple segments. Show less

An apparatus of includes a processor component to: transmit node device identifiers to multiple node devices to define an ordering thereamong and among subsets of multiple blocks of data distributed thereamong; receive sizes of the subsets from the multiple node devices; derive block exchanges among the multiple node device based on the sizes and a minimum size imposed on data transmissions to storage device(s); and transmit a block exchange vector that describes the block exchanges to the… Show more

An apparatus of includes a processor component to: transmit node device identifiers to multiple node devices to define an ordering thereamong and among subsets of multiple blocks of data distributed thereamong; receive sizes of the subsets from the multiple node devices; derive block exchanges among the multiple node device based on the sizes and a minimum size imposed on data transmissions to storage device(s); and transmit a block exchange vector that describes the block exchanges to the multiple node devices, wherein: the subsets remain distributed among a reduced number of the multiple node devices following the block exchanges; at least all node devices of the reduced number but one stores an amount of the blocks of data exceeding the minimum size; and the block exchanges are all lower-order to higher-order node device transfers, or all higher-order to lower-order node device transfers. Show less

An apparatus including a processor to index data records within a data cell, wherein for each data record, the processor retrieves data values from first and second data fields; determines whether the first and second data fields store unique data values; in response to the first data field storing a unique data value, adds an identifier of the data record to a first unique values index, in response to the second data field storing a unique data value, adds the identifier to a second unique… Show more

An apparatus including a processor to index data records within a data cell, wherein for each data record, the processor retrieves data values from first and second data fields; determines whether the first and second data fields store unique data values; in response to the first data field storing a unique data value, adds an identifier of the data record to a first unique values index, in response to the second data field storing a unique data value, adds the identifier to a second unique values index, wherein identifiers of data records within the unique values indexes are ordered based on corresponding unique data values; and generates an indication of ranges of data values of the first and second data fields to enable a determination of whether a data value specified in search criteria is present within at least the data cell. Show less

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meetingthe search criteria, and in response to a determination that the data cell includes such a data… Show more

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meetingthe search criteria, and in response to a determination that the data cell includes such a data record, use a unique values index in the cell index to search the data records of the data cell to identify one or more data records meeting the search criteria; and in response to identifying at least one data record meeting the search criteria, provide an indication that at least the data cell includes at least one data record meeting the search criteria.
Show less

An apparatus may include a processor component caused to: generate map entries in map data descriptive of encrypted data blocks within a data file; use first map block encryption data to encrypt a first map extension of the map data; transmit the encrypted first map extension for storage within the data file; store the first map block encryption data within the second map extension; use second map block encryption data to encrypt a second map extension of the map data after storage of the first… Show more

An apparatus may include a processor component caused to: generate map entries in map data descriptive of encrypted data blocks within a data file; use first map block encryption data to encrypt a first map extension of the map data; transmit the encrypted first map extension for storage within the data file; store the first map block encryption data within the second map extension; use second map block encryption data to encrypt a second map extension of the map data after storage of the first map block encryption data therein; transmit encrypted second map extension for storage within the data file; store the second map block encryption data within the map base; use third map block encryption data to encrypt a map base of the map data after storage of the second map block encryption data therein; and transmit the encrypted map base for storage within the data file. Show less

An apparatus includes a processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indications of which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for each map… Show more

An apparatus includes a processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indications of which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for each map data map entry: retrieve a hashed identifier for a data sub-block, and a size for each of the data sub-blocks within the corresponding data block; divide the hashed identifier by the quantity of available node devices; compare the modulo value to a designation assigned to each of the available node devices; and provide a pointer to the available node device assigned the matching designation. Show less

An apparatus includes a processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indicationsof which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for cacti map… Show more

An apparatus includes a processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indicationsof which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for cacti map data map entry: retrieve a hashed identifier for a data sub-block, and a size for each of the data sub-blocks within the corresponding data block; divide the hashed identifier by the quantity of available node devices; compare the modulo value to a designation assigned to each of the available nodedevices; and provide a pointer to the available node device assigned the matching designation.
Show less

An apparatus including a processor to: index multiple data records within a data cell by first and second data fields in a single read pass through the data cell; wherein for each data record within the first data cell, the processor is to retrieve data values from the first and second data fields, search a first binary tree to determine whether the data value from the first data field comprises a unique value, and add the data value to the first binary tree if it is unique, and search a second… Show more

An apparatus including a processor to: index multiple data records within a data cell by first and second data fields in a single read pass through the data cell; wherein for each data record within the first data cell, the processor is to retrieve data values from the first and second data fields, search a first binary tree to determine whether the data value from the first data field comprises a unique value, and add the data value to the first binary tree if it is unique, and search a second binary tree to determine whether the data value from the second data field comprises a unique value, and add the data value to the second binary tree if it is unique; and generate a first and second unique values indexes of identifiers of the data records associated with the unique data values within the first and second binary trees. Show less

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meeting the search criteria, and in response to a determination that the data cell includes such a data… Show more

An apparatus including a processor to receive search criteria including a data value for a search within a data field; in response to the receipt of the query instructions, and for each data cell within a super cell, perform the specified search by comparing the data value to ranges of values indicated in a corresponding cell index to determine whether the data cell includes a data record meeting the search criteria, and in response to a determination that the data cell includes such a data record, use a unique values index in the cell index to search the data records of the data cell to identify one or more data records meeting the search criteria; and in response to identifying at least one data record meeting the search criteria, provide an indication that at least the data cell includes at least one data record meeting the search criteria. Show less

An apparatus including a processor caused to: receive sizes and data block encryption data for multiple encrypted data blocks from multiple node devices, wherein data block encryption data is separately generated and used by each node device to encrypt a portion of a data set to generate one of the multiple encrypted data blocks; for each encrypted data block, generate a corresponding map entry within map data to include size and data block encryption data; and in response to receiving size and… Show more

An apparatus including a processor caused to: receive sizes and data block encryption data for multiple encrypted data blocks from multiple node devices, wherein data block encryption data is separately generated and used by each node device to encrypt a portion of a data set to generate one of the multiple encrypted data blocks; for each encrypted data block, generate a corresponding map entry within map data to include size and data block encryption data; and in response to receiving size and data block encryption data for all encrypted data blocks, encrypt a portion of the map data to generate an encrypted map base, wherein the portion of map data includes at least a subset of the multiple map entries, and transmit the encrypted map base to one or more storage devices to be stored within a data file along with the multiple encrypted data blocks. Show less

An apparatus includes processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indications of which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for each map… Show more

An apparatus includes processor component caused to: retrieve metadata of organization of data within a data set, and map data of organization of data blocks within a data file; receive indications of which node devices are available to perform a processing task with a data set portion; and in response to the data set including partitioned data, compare the quantities of available node devices and of the node devices last involved in storing the data set. In response to a match, for each map data map entry: retrieve a hashed identifier for a data sub-block, and a size for each of the data sub-blocks within the corresponding data block; divide the hashed identifier by the quantity of available node devices; compare the modulo value to a designation assigned to each of the available node devices; and provide a pointer to the available node device assigned the matching designation. Show less

Division of US8555378 B2 with a new claim set representing cache synchronization

Techniques to explain authorization origins for protected objects in an object domain are disclosed. In one embodiment, for example, an apparatus may comprise a processor circuit, a request processor component operative on the processor circuit to receive and process a request for an authorization origin of a resource object, the authorization origin comprising an access control with a permission arranged to control access to the resource object based on an identity, and a resource origin… Show more

Techniques to explain authorization origins for protected objects in an object domain are disclosed. In one embodiment, for example, an apparatus may comprise a processor circuit, a request processor component operative on the processor circuit to receive and process a request for an authorization origin of a resource object, the authorization origin comprising an access control with a permission arranged to control access to the resource object based on an identity, and a resource origin component operative on the processor circuit to identify the authorization origin of the resource object from a set of interrelated resource objects and associated access controls, retrieve authorization origin information for the authorization origin, and present the authorization origin information in a user interface view. Other embodiments are described and claimed. Show less

Systems and methods are included for accessing resource objects in a multi-threaded environment. A request is received from a requester to perform an operation with respect to a resource object, where the requested resource object has multiple associations with other objects. A determination as to whether an authorization cache entry corresponding to the requested resource object contains sufficient permission data for granting or denying the request for access to the requested resource object… Show more

Systems and methods are included for accessing resource objects in a multi-threaded environment. A request is received from a requester to perform an operation with respect to a resource object, where the requested resource object has multiple associations with other objects. A determination as to whether an authorization cache entry corresponding to the requested resource object contains sufficient permission data for granting or denying the request for access to the requested resource object is made. A grant or deny of access to the requested resource object is returned when the authorization cache entry corresponding to the requested resource object contains sufficient permission data. Show less

Computer-implemented systems and methods for determining whether to authorize one or more operations with respect to resource objects. A system and method can include receiving a request that would involve an operation with respect to a resource object. The requested resource object may have multiple associations with other objects. One or more data stores are used to store interrelationships among resource objects, authorization-related objects, and access permission information. A comparison… Show more

Computer-implemented systems and methods for determining whether to authorize one or more operations with respect to resource objects. A system and method can include receiving a request that would involve an operation with respect to a resource object. The requested resource object may have multiple associations with other objects. One or more data stores are used to store interrelationships among resource objects, authorization-related objects, and access permission information. A comparison is performed using permissions and the requester's access credential information. The comparison is used to determine whether to permit the operation with respect to the resource object. Show less