File loading suggestions and issues
Storage types
Especially for things like DataHub
| Type | Reliability | Comments |
|---|---|---|
| Direct local directory | 👍Very high | This provides us with the tools to know when the file can be locked preventing changes. |
| SFTP | 👍Very high | This provides us with the tools to know when the file can be locked preventing changes. |
| 👋High | Once it arrives, it is very reliable. | |
| Sharepoint | 👍Very High | Login issues, but once you are connected, if we get a file it is 'completed' (no changes pending) |
| Virtual File System | 👋High | This provides us with the tools to know when the file can be locked preventing changes. |
| FTPS | ⚠️Medium | While it has an S like SFTP, FTPS is very different and it doesn't provide the same level of features that SFTP does. It is possible to read a file, unaware that the sender wasn't finished writing to it. We recommend against using FTPS. |
| FTP | 🛑Low | Errors can occur on transmission, no way for us to be 'sure' the file is complete - it gets bigger while things are being written to it, if no changes for a period of time, typically 10 seconds, it is treated as complete. We therefore strongly recommend against using plain FTP. |