Spark Spill to Disk: Why It Happens and How to Fix It
Spill means Spark ran out of memory in the middle of a task and dumped its working data to disk. Your job does not fail — it just gets slower, with nothing in the logs to explain why.
Read MoreSpill means Spark ran out of memory in the middle of a task and dumped its working data to disk. Your job does not fail — it just gets slower, with nothing in the logs to explain why.
Read More