Ok, thanks a lot Demerit but I could not get what I should do in there and finally found the solution.
So the trick was with the capabilities.
I installed user role editor and found out that by default, a subscriber is granted way more capabilities than just "read", which are :
Go to https://your-site/wp-admin/admin.php?page=CiviCRM&q=civicrm%2Fadmin%2Faccess%2Fwp-permissions&reset=1
You can also get there from under administer - users and permissions - permissions, and then on that page there's the link to wordpress access control.
After having tested all the possibilities by checking and unchecking all the options marked in the WordPress Access Control pane, I found that the box "CiviCRM: access CiviCRM backend and API" must be be ticked. Otherwise you dont have access to CiviCRM. Strange because it is told to give this to trusted roles only. The well done documentation is ...