The Line Item table is oriented towards the current state of an order's line item. The Financial Item table is oriented towards keeping track of the history of the financial aspects of line items. The Financial Transaction table is the base table for financial transactions created by CiviCRM every time there is any add, update, cancel, refund, pay grant or other action that has a financial impact.
At a high level, each time there is a change in a line item that produces a financial effect, like a change in the quantity, or the price, or the financial type associated with the line item, a reversal entry of the previous financial impact of the line item is made in the financial item table, and a new entry is inserted representing the current financial impact.
Another difference is that line items are more like business entities while financial items are aspects of financial accounting entries. To hide the complexity of double entry bookkeeping, CiviCRM uses Financial Types to create a bundle of Financial Accounts that will be used through the life cycle of a transaction for things like its Accounts Receivable account, its Revenue account, and potentially other accounts like the Accounts Payable or Contra Revenue account for refunds owing. These financial accounts are the ones that appear in the Chart of Accounts for an organization. Line items have financial types associated with them, while the specific detailed nature of financial items means they have financial accounts associated with them.
The civicrm_financial_trxn and civicrm_financial_item tables hold the main accounting or bookkeeping information in CiviCRM while civicrm_entity_financial_trxn is a related table that connects records in civicrm_financial_trxn to related records in various other tables. There are many types of financial transactions, and they all are stored in these tables, and are exportable via Financial Batch exports to Accounting Packages like QuickBooks, Sage, and Microsoft Dynamics.
Bookkeeping entries have two sides to them: a debit account and a credit account, as well as an amount and a date. Some bookkeeping entries are split, where a single amount on one side is split amongst several amounts and accounts on the other side. For example, a payment for an event registration and a donation would have the total going into the bank on one side, and revenue being recognized for two accounts on the other side.
In the case of contributions which handle the finances for donations, memberships and event registration as well as other sale items, each line item will create a civicrm_financial_item, each payment will create a civicrm_financial_trxn, and civicrm_entity_financial_item records will connect them. In the case of line items with sales tax, which in general goes to a different account in an organization's chart of accounts, there will be an additional entry in the civicrm_financial_item table representing the tax on a line item.
An important principle of accounting is that books should be auditable. One part of that is to retain records about all of the changes that have been made on items. So if a purchase is made with pay later selected, and two partial payments are made, the number of things purchased is changed, a refund is made, etc., then all of these are tracked in the financial tables in CiviCRM. Entries in civicrm_financial_item and civicrm_financial_trxn tables are intended to never be edited or deleted.
We highly discourage developers trying to write directly to the civicrm_financial_trxn, civicrm_financial_item and civicrm_entity_financial_trxn tables which store financial information in CiviCRM since version 4.3. We'd encourage use of the order api and/or payment api, and will enhance and/or fix issues with those APIs.
Documentation that is reasonably accurate on entries for these interlinked tables for various types of accounting transactions can be found at https://wiki.civicrm.org/confluence/display/CRM/CiviAccounts+Data+Flow#CiviAccounts4.3DataFlow-OnlineorofflinecontributionwithFEE.
The Caldera Forms CiviCRM plugin for WordPress was recently trying to integrate its support for various line item operations with Civi core, so their code may provide useful guidance. The Line Item Edit extension might also be a useful developer resource, though I seem to recall that it needs a bit of refactoring.