This can be costly for limited use images, but worth it for those you might use repeatedly like a good rubber stamp set. I like to print these on tissue paper cut and secured to cardstock or on white cardstock. I cut them apart and store in a large bin I can dig through when designing a canvas.
But, my favorite kind of ephemera is the vintage or re-purpossed items you find digging in thrift stores, used book stores or at garage/tag sales.
I like to use neutral items, like pattern directions, old book pages, maps & bargain scrapbook papers when I begin to collage my canvases. I also like to use these with my gelli plate. A $2 dictionary from a clearance shelf will go along way as will $1 sewing patterns (size and style do not matter!).
Check with friends and family...they might love for someone to haul off a set of old encyclopedias. Keep the Index book to source specific topics, color picture plates and maps.
Swap stamped images, die cuts or parts of a set you don't need or want with friends or organize a mixed media crop where everyone brings something to work on and stuff to share.