Videos and Photos become distorted after transfer?

[kabutiah]

I’ve got a triple system build into the same PC case:
- Windows System
- Hackintosh System
- NAS System

My NAS System is a LattePanda 3 Delta with Windows Server 2016 installed.
The storage drives are NVMe SSDs.

All 3 systems are hooked up together via Ethernet onto a QNAP QSW-1105-5T.
I can connect and access the files instantly, but the issue starts with the transfer of high res photos and videos.

Whenever I attempt to transfer any of these photo and video files from say the Windows System into the NAS system, the photo and video becomes distorted (blurry, colourful lines, jagged).

Transferring an existing video or photo from the NAS system to the Windows system doesn’t have the same problem.

Does anyone know why this happens and how I can fix it?

Thanks in advance!

 

[drinkingbird]

I’ve got a triple system build into the same PC case:
- Windows System
- Hackintosh System
- NAS System

My NAS System is a LattePanda 3 Delta with Windows Server 2016 installed.
The storage drives are NVMe SSDs.

All 3 systems are hooked up together via Ethernet onto a QNAP QSW-1105-5T.
I can connect and access the files instantly, but the issue starts with the transfer of high res photos and videos.

Whenever I attempt to transfer any of these photo and video files from say the Windows System into the NAS system, the photo and video becomes distorted (blurry, colourful lines, jagged).

Transferring an existing video or photo from the NAS system to the Windows system doesn’t have the same problem.

Does anyone know why this happens and how I can fix it?

Thanks in advance!

Unlikely to be the network as TCP has plenty of measures built in to prevent corruption and errors. Sounds like you may have a bad drive in your NAS or bad driver etc.

 

[kabutiah]

Unlikely to be the network as TCP has plenty of measures built in to prevent corruption and errors. Sounds like you may have a bad drive in your NAS or bad driver etc.

I’m not sure if I could possibly rule out bad drive as the issue doesn’t occur if I plugged a USB directly into the LattePanda (NAS system).
There are also 2 different NVME SSDs on it that both have data corruption issues over the network (unless by sheer bad luck, they are both bad drives).

What do you mean by bad driver in this context?
Which device would I need to update the firmware?
Thanks.

 

[drinkingbird]

I’m not sure if I could possibly rule out bad drive as the issue doesn’t occur if I plugged a USB directly into the LattePanda (NAS system).
There are also 2 different NVME SSDs on it that both have data corruption issues over the network (unless by sheer bad luck, they are both bad drives).

What do you mean by bad driver in this context?
Which device would I need to update the firmware?
Thanks.

Either the NVMe or network driver on the NAS would be the first place I'd look. If you have a switch, you could plug it between the NAS and router and plug a PC into the same switch, try transferring a file and see if it still happens. If so it is either the PC or NAS, you've ruled out the router completely at that point, and if it happens with multiple PCs, almost certainly the NAS. If the corruption only happens over the network, I'd check the network driver on the NAS. Even if it was a bad cable, it should not be corrupting files, just transferring really slow.

Could be an issue with the software running on the NAS too, that's one of the challenges with homebrew stuff.

If the corruption only happens when going through the router, need to do some more troubleshooting to narrow it down further (does it happen wired and wireless, multiple PCs, etc).

Are the two drives runing in RAID0? If so, a single bad drive would still corrupt everything since the data is written across both. However if it doesn't happen when transferring from USB that seems less likely.

 

[kabutiah]

Could be an issue with the software running on the NAS too, that's one of the challenges with homebrew stuff.

Actually, it’s not running any software at all. Just the regular file sharing over network from File Explorer.

Over the network:
- If I played a video already in the NAS system (transferred from a USB), it plays perfectly fine.

- If I transferred a video from the NAS system into the Windows system and played the video from the Windows system, it plays perfectly fine.

- If I transferred a video from the Windows system to the NAS system and played it (exactly as the first example), the video becomes blurry and jagged like it was corrupted.

 

[drinkingbird]

Actually, it’s not running any software at all. Just the regular file sharing over network from File Explorer.

Over the network:
- If I played a video already in the NAS system (transferred from a USB), it plays perfectly fine.

- If I transferred a video from the NAS system into the Windows system and played the video from the Windows system, it plays perfectly fine.

- If I transferred a video from the Windows system to the NAS system and played it (exactly as the first example), the video becomes blurry and jagged like it was corrupted.

If the Windows and NAS are both wired to a switch, very unlikely to be network hardware. You can test that by running them through another switch, or connecting them directly with a crossover cable and setting static IPs. But from your symptoms above, something is wrong with your NAS, possibly the NIC and/or driver, possibly the connection between the NIC and the drives (backplane). For kicks you can try swapping cables (make sure cat5e or better, preferably CAT6+) but the network stack is designed to detect errors due to bad cables and correct.

What happens if you transfer from windows to a USB on the NAS directly? That would help determine if it is related to the drives/NVMe subsystem or not.

 

[kabutiah]

If the Windows and NAS are both wired to a switch, very unlikely to be network hardware. You can test that by running them through another switch, or connecting them directly with a crossover cable and setting static IPs. But from your symptoms above, something is wrong with your NAS, possibly the NIC and/or driver, possibly the connection between the NIC and the drives (backplane). For kicks you can try swapping cables (make sure cat5e or better, preferably CAT6+) but the network stack is designed to detect errors due to bad cables and correct.

What happens if you transfer from windows to a USB on the NAS directly? That would help determine if it is related to the drives/NVMe subsystem or not.

All Ethernet cables used are short 50cm CAT8 cables.

I’ve just tested this:
- Plugged USB straight into NAS system (LattePanda). Transferred video file from Windows system through the network into the USB drive. Played video file from USB drive over the network. Exact same results with messed up video.

- Plugged USB straight into NAS system (LattePanda). Transferred video file from NAS system (already existed there) into USB drive. Played video file from USB drive over the network. Video played perfectly fine.


Also, worth noting that if I play the same “corrupted” video on the NAS system itself (not on my Windows system via network), the video still looks messed up.

So I think it looks like there is definitely something that happens to the file itself whenever any sort of digital file gets transferred through the network.

One last note, .doc or .pdf files transfer between the network perfectly fine.

 

[drinkingbird]

All Ethernet cables used are short 50cm CAT8 cables.

I’ve just tested this:
- Plugged USB straight into NAS system (LattePanda). Transferred video file from Windows system through the network into the USB drive. Played video file from USB drive over the network. Exact same results with messed up video.

- Plugged USB straight into NAS system (LattePanda). Transferred video file from NAS system (already existed there) into USB drive. Played video file from USB drive over the network. Video played perfectly fine.


Also, worth noting that if I play the same “corrupted” video on the NAS system itself (not on my Windows system via network), the video still looks messed up.

So I think it looks like there is definitely something that happens to the file itself whenever any sort of digital file gets transferred through the network.

One last note, .doc or .pdf files transfer between the network perfectly fine.

What if you transfer the file from the hackintosh to the NAS?

If same corruption I'd really start focusing on the NAS, check network drivers and settings, etc.

Also may be worthwhile to connect a computer to it with crossover cable directly just to rule out your switch but the switch seems very unlikely unless it has some major flaws. You could also rule it out potentially by transferring the file between the windows and hackintosh machines.

 

[kabutiah]

What if you transfer the file from the hackintosh to the NAS?

If same corruption I'd really start focusing on the NAS, check network drivers and settings, etc.

Also may be worthwhile to connect a computer to it with crossover cable directly just to rule out your switch but the switch seems very unlikely unless it has some major flaws. You could also rule it out potentially by transferring the file between the windows and hackintosh machines.

I couldn’t test it on the Hackintosh due to the newest Ventura OS update completely ruining network sharing.

I did, however, use a super long Ethernet cable to test it with another Windows PC in another room.
Transfer speeds were extremely fast.

I did all the same testing and got the exact same results with the corrupted media files.

Just wondering, how would I update or check the drivers of my QNAP because it’s only a switch?
Unless you’re referring to the NAS system (LattePanda) that is running Windows Server 2016?

Thanks for your help and advice. Appreciate it.

 

[drinkingbird]

I couldn’t test it on the Hackintosh due to the newest Ventura OS update completely ruining network sharing.

I did, however, use a super long Ethernet cable to test it with another Windows PC in another room.
Transfer speeds were extremely fast.

I did all the same testing and got the exact same results with the corrupted media files.

Just wondering, how would I update or check the drivers of my QNAP because it’s only a switch?
Unless you’re referring to the NAS system (LattePanda) that is running Windows Server 2016?

Thanks for your help and advice. Appreciate it.

Yeah I mean drivers on the NAS box, specifically the network drivers, possibly the storage or chipset drivers.

 

[degrub]

here is an out of the box idea - take one of the damaged files and do all of the transfer and view tests - same result ?

 

[kabutiah]

here is an out of the box idea - take one of the damaged files and do all of the transfer and view tests - same result ?

Yea same results.

Transferring any media files from into the NAS via the network seems to permanently alter the file.

 

[sfx2000]

But from your symptoms above, something is wrong with your NAS, possibly the NIC and/or driver, possibly the connection between the NIC and the drives (backplane). For kicks you can try swapping cables (make sure cat5e or better, preferably CAT6+) but the network stack is designed to detect errors due to bad cables and correct.

I tend to agree here - are the files truly corrupt? download from NAS back to Local - if they are still... then there is a good change it could be storage related, or it's corrupted across the wire perhaps

 

[tgl]

This is jogging some ancient memories from my days as a graphics-file-format hacker ...

These symptoms sound a whole lot like the kind of corruption we used to see in image files if something decided that it needed to apply a Windows<->Unix newline conversion, that is LF to CR+LF or vice versa. If the corruption is completely repeatable in multiple attempts with the same file, then I think you want to discard the notion of flaky network operation and start looking for effects like misguided conversions. Unfortunately I don't do enough with Windows to be able to point you at exactly what settings could need to be changed.

 

[drinkingbird]

This is jogging some ancient memories from my days as a graphics-file-format hacker ...

These symptoms sound a whole lot like the kind of corruption we used to see in image files if something decided that it needed to apply a Windows<->Unix newline conversion, that is LF to CR+LF or vice versa. If the corruption is completely repeatable in multiple attempts with the same file, then I think you want to discard the notion of flaky network operation and start looking for effects like misguided conversions. Unfortunately I don't do enough with Windows to be able to point you at exactly what settings could need to be changed.

If it is just file transfers there shouldn't be any of those conversion issues going on, regardless of the NAS they all support standard protocols for file transfer. Guess anything is possible but I suspect it is on the NAS end not windows.

 

[sfx2000]

If it were Samba, this would be easy...

That being said - it could be some service (indexer) running... maybe adding tags/metadata to the files.

Interesting thread here, apparently this is a common thing with Solitaire Shell (Windows)

mp4 file corruption

social.technet.microsoft.com

 

[tgl]

That being said - it could be some service (indexer) running... maybe adding tags/metadata to the files.

Hmm, maybe. That would be more plausible if the corruption didn't appear right away, but only after the file had sat on the NAS for awhile.

Anyway, we have multiple competing theories and no data to prove anything one way or the other. I suggest the following experiments:

* Copy one of the vulnerable files to the NAS and then back again. Compare the results. Is the corrupted file longer, shorter, or same-length-but-different-contents?

* Do that several times. Are the corrupted copies all identical, or do they vary in content?

* Repeat for a few different files, see if your answers are consistent.

 

[drinkingbird]

Hmm, maybe. That would be more plausible if the corruption didn't appear right away, but only after the file had sat on the NAS for awhile.

Anyway, we have multiple competing theories and no data to prove anything one way or the other. I suggest the following experiments:

* Copy one of the vulnerable files to the NAS and then back again. Compare the results. Is the corrupted file longer, shorter, or same-length-but-different-contents?

* Do that several times. Are the corrupted copies all identical, or do they vary in content?

* Repeat for a few different files, see if your answers are consistent.

Also the fact that corruption doesn't happen when copying from USB, only over the network. Don't know why a windows based NAS would make any changes that would corrupt an MP4 file, even if it did add metadata, etc. It knows how to deal with those files. The MS discussion quoted is also a very old windows 7 one which had some known 4Gb file size issues and those were addressed long ago.

 

[sfx2000]

Hmm, maybe. That would be more plausible if the corruption didn't appear right away, but only after the file had sat on the NAS for awhile.

Could be client side - Windows mediaplayer can add tags to content.

Going back to the top of the thread, the issue only pops up when the windows client is accessing the data - his other platforms seem to be fine.