What is the best practice between CiviReport and Views ? (Except the fact than Views enables to order the columns, which CiviReport does not)
Why and when should we use one rather than the other ?
A basic difference is that CiviReport is CMS-independent whereas Views is a Drupal module so those running WordPress or Joomla don't have that choice. (Yes, OP knows that ... some readers might not!)
In part, it depends on who is setting up the reports/views:
For end-users, using the configuration tabs to adjust the output is easier than adding fields and filters via the Views UI. But the simplicity has limitations - you can't determine the order of fields in a report, you can only select the fields that have made available by the report author etc.
For site-builders, Views offer more configurability without code.
For developers, custom reports can make it easier for end-users to tweak their own reports and code-level data manipulation allows more complex presentation of data.