It depends on how you use product hierarchy.
As you noted, since product hierarchy is captured in the sales documents then they will also need to be updated. It might also be copied to the subsequent documents. Also if are using any LIS tables based on product hierarchy then those need to be updated as well. Some BI/BW update might be needed, if using.
However, in some cases it's not necessary at all to update the sales documents and instead the reports can be adjusted. E.g. in similar situations in the past we changed several reports/queries to look up hierarchy from the material master instead of the documents. It's a bit less efficient but spared us from constant "realignments".