I don't see a way of doing it without writing custom code
One path would be to create a custom search that has the right sql to contain only the most recent one. This is closest to what you want, but might have a performance impact (eg if you have loads of records/contacts) when civi builds the smart group
Another one is to denormalise and create the same custom fields you have on the activity but add them (as read only) on the contact instead of the activity
and then you write a hook so everytime a new activity is created with these custom fields, it would automatically duplicate them to the contact... or skip the duplicate custom fields on contact and directly add them to the right group if it's your only use case.
So the activities contain all the history, but the latest version is always stored in the contact (or belong to the right group)
This might get a little bit more tricky if you have to handle edge cases (eg what happens if you delete an activity? if you merge two contacts that have activities?)
This might or might not be an issue in your case
Another way to skin that cat would be to improve civirule to allow new triggers on custom fields.
Good luck, writing an extension isn't that complicated if you know php/sql