I've investigated this myself, but haven't had to do it yet. Broadly, you would use the
GROUP_CONCAT MySQL function. You can see examples of this being used on core contribution reports for displaying multiple credit card types for contributions paid in multiple installments with different cards.
I also have a fork of the salutations extension that provides a UI for exporting multiple-record custom groups as part of an export. Instead of putting all the data in a single cell, I break the data out into multiple columns, but all on the same row.
You could also use the SQL Tasks extension to generate the report using pure SQL, if you need a quicker solution and can trade off some configurability.