Hi,
When doing iterative computing, I found the lineage on IndexedRDDs cannot be cut through checkpointing. It is because the newRDD returned by doCheckpoint() is a CheckpointRDD.
I'm presently using IndexedRDD as a Spark package. Is there any way other than checkpoint (through public API) that I can use to cut the lineage? Thank you! Any suggestion is appreciated...
Hi,
When doing iterative computing, I found the lineage on IndexedRDDs cannot be cut through checkpointing. It is because the newRDD returned by doCheckpoint() is a CheckpointRDD.
I'm presently using IndexedRDD as a Spark package. Is there any way other than checkpoint (through public API) that I can use to cut the lineage? Thank you! Any suggestion is appreciated...