My understanding is that the rel="canonical" is intended only for pages with 100% duplicate content. In the scenario you describe you've actually got different content on each of those pages, and the URL querystring ?pagenum=n controls which set of content is being displayed.
I don't think that the canonical tag is appropriate for this, but rel="next" and rel="prev" probably are.
You should do a View All page if you can without taking too big a performance hit on that page load. Then put a canonical pointing to the View All on each of the paginated entities. If a View All just isn't possible, then just leave off the canonical altogether.
Those paginated entities should also have rel="next" and rel="prev" tags in their heads, pointing to the next and previous pages in the series. The first page in the series will not have a rel="prev" and the last page won't have a rel="next".