Optional engine-specific all reduce tensor operation.
Engine-specific colMeans implementation based on a checkpoint.
Engine-specific colSums implementation based on a checkpoint.
Convert non-int-keyed matrix to an int-keyed, computing optionally mapping from old keys to row indices in the new one.
Convert non-int-keyed matrix to an int-keyed, computing optionally mapping from old keys to row indices in the new one. The mapping, if requested, is returned as a 1-column matrix.
Broadcast support
Broadcast support
Load DRM from hdfs (as in Mahout DRM format).
Load DRM from hdfs (as in Mahout DRM format). <P/>
The DFS path to load from
Minimum parallelism after load (equivalent to #par(min=...)).
This creates an empty DRM with specified number of partitions and cardinality.
Creates empty DRM with non-trivial height
Parallelize in-core matrix as the backend engine distributed matrix, using row ordinal indices as data set keys.
Parallelize in-core matrix as the backend engine distributed matrix, using row labels as a data set keys.
(Optional) Sampling operation.
(Optional) Sampling operation. Consistent with Spark semantics of the same.
Load IndexedDataset from text delimited format.
Load IndexedDataset from text delimited format.
comma delimited URIs to read from
defines format of file(s)
Load IndexedDataset from text delimited format, one element per line
Load IndexedDataset from text delimited format, one element per line
comma delimited URIs to read from
defines format of file(s)
Engine-specific numNonZeroElementsPerColumn implementation based on a checkpoint.
Second optimizer pass.
Second optimizer pass. Translate previously rewritten logical pipeline into physical engine plan.
First optimization pass.
First optimization pass. Return physical plan that we can pass to exec(). This rewrite may introduce logical constructs (including engine-specific ones) that user DSL cannot even produce per se. <P>
A particular physical engine implementation may choose to either use the default rewrites or build its own rewriting rules. <P>
Abstraction of optimizer/distributed engine