Question: Migrating Existing Site to Use External Storage or CDN

We recently implemented an NFS Server + Client setup, which successfully expanded our disk space without downtime — and the change is permanent. It’s been a real lifesaver under the circumstances.

That said, I do have a technical question for the UNA team or anyone with experience in this area:

As we know, CDN and external storage solutions (like AWS S3, Azure Blob, or other CDN vendors) are commonly used to handle media files and improve performance.
In our case, the platform wasn’t originally configured to separate /storage or media directories to an external source. Given that, what are the odds or best practices for migrating an existing UNA site — already in production — to use external or CDN-based storage instead of local disk?

I understand this wasn’t the ideal setup from the start, but now that the system is stable again, we’d like to explore more scalable and performance-oriented options going forward.

Any guidance, examples, or recommended steps for doing such a migration safely would be greatly appreciated.

Best regards,

Chris

  • View
    444
  • More
Replies (7)

    • Hi Guys,

      Please urgently respond if you have any similar experience or might need to migrate to cdn storage type in the near future due to a bad start by using local storage never to know that you'll succeed,

      • First of all, you have to compare in your head what you really want and what you expect from it. How many users will you have, what hardware will you use, etc. For example, if you will process an enormous amount of videos, no VPS server will be enough for you. Many providers do not give root access to VPS, which is also a problem if you need to set something up quickly. You have to wait and rely on strangers all the time. If you have your own big project, buy your own refurbished server with warranty and use server housing like I do.

        This will save you a lot of worries in the future, the server performance will be incomparable with a VPS, and as an additional bonus, you can add disks as needed.

        Good luck with your choice :)

        • Hey @PavelS .. Your response means a lot. If I understand you correctly.. This is what one should be doing right from the start though.

          What I really want to know is how you will split or migrate a standard unacms/storage site to a cdn type of storage for your media files.

          We currently don't have an export tool for purely data or lets say, an export tool for videos and images including a tool to transfer it to the new.

          We now sit with over 400G of /storage and feel stuck, even-though we connected another drive of 300G using NFS methods

          • Do you want to use a CDN to speed up the loading of videos and images while having unlimited storage outside your own machine? Technically, it’s possible to do, but it’s not entirely simple.

            UNA by default stores all files (photos, videos, attachments, etc.) in the /storage folder on the same server where the website runs.

            To move this to a CDN or another remote storage, new files must start uploading to the new location (for example S3).

            Old files must be copied there as well, and the system’s links must point to the new source — so that files are now loaded from the CDN (e.g. https://cdn.domain.com/storage/...).

            You do need to physically transfer the data and adjust the configuration so the website knows where to read them from.

            If I'm wrong, someone will definitely correct me.

            • There are some instructions on how to migrate to the remote S3 storage:

              https://unacms.com/wiki/Remote-Storage

              You can use `aws s3` command to transfer local files to the s3 bucket, this example is to be run from `storage` folder in UNA:

              aws s3 . s3://mybucket --acl "public-read"

              If you are using non Amazon S3 storage you can use the same command with `endpoint-url` parameter:

              aws s3 --endpoint-url=https://s3.us-east-2.wasabisys.com sync . s3://mybucket --acl "public-read"

              If you have a lot of data you can speedup using custom parameters, some example values:

              aws configure set default.s3.max_concurrent_requests 24
aws configure set default.s3.max_queue_size 10000
aws configure set default.s3.multipart_threshold 64MB
aws configure set default.s3.multipart_chunksize 16MB
aws configure set default.s3.max_bandwidth 1024MB/s
aws configure set default.s3.use_accelerate_endpoint true

              • Hey Guys @Alex T⚜️ , @PavelS ... Just got the one notification and immediately respond ... reading only a bit and was beyond excited to know that indeed, it is possible to migrate a standard unacm site that has grown into, if I can say impossible task considering the amount of records we currently sitting with... and of course getting real worried for we might have to face, you know what I mean... thank you very much and will investigate immediately...

                • For information and guidance purposes - the following threat is an extension to the overall storage challenges as well as management thereof.

                  https://unacms.com/d/best-way-to-clean-up-old-media-files

                  I once again would like to thank all parties participated in this discussion that has made a significant difference in the challenges we faced, never to know that our answer was actually right in front of us...

                  Login or Join to comment.