Thought that title might grab your attention, but beyond a respect for the great mammals, and more than a few laws preventing you from even attempting it, we did no such thing as riding whales.
Our journey to the remote East Cape of the north island took us past swan filled lakes to Gisbourne, the last civilization for a few hundred kilometres. It's also the site of Cook's landing at New Zealand, with a couple of statues marking the fact, even though it's now obscured by a small container port.
The area is also the setting to a Maori story recently told in the film Whale Rider, where a girl from a tribe fights against adversity to prove that she is the successor to the chief of the tribe whose ancestor arrived in New Zealand on the back of a whale. A good film, and a little unusual from the norm, along with teaching a bit about the Maori culture - watch it if you can!