Morning will this help you at all https://unacms.com/wiki/Video-Transcoder

In this architecture, video storage is separated from the main application by hosting it on a dedicated server or NAS cluster. This centralized "storage folder" serves as a repository for all video files and is accessible to both the main UNA site and multiple video transcoding servers. This separation ensures that heavy processing tasks like video transcoding are offloaded from the primary application, thereby improving overall performance and scalability.

Key Architectural Components

Dedicated Video Storage

All video files are stored in a centralized folder on a separate server or NAS cluster. This approach enables high availability and centralized management of video assets. Both the main UNA application and dedicated video transcoding servers can consistently access this shared storage. Several storage solutions are available, depending on your needs for scalability, performance, and availability:

• NFS (Network File System)
• Description: A simple and widely used protocol for sharing directories among multiple Linux servers.
• Advantages: Easy to set up; works with most Linux distributions; allows multiple servers to mount the same folder; scalable by adding additional NFS servers.
• Disadvantages: May struggle under heavy I/O loads or with many clients; not optimized for storing very large files.
• Recommended Use: Suitable for small to medium applications with moderate performance needs.
• CIFS/SMB (Common Internet File System)
• Description: Commonly used in Windows environments and also available on Linux via cifs-utils.
• Advantages: Simple configuration in hybrid environments; supports user-based access control.
• Disadvantages: Performance on Linux may be lower compared to NFS; can be more complex to configure for high-demand setups.
• Recommended Use: Ideal for environments that combine both Windows and Linux servers.
• OpenStack Manila (Shared File Storage)
• Description: Manila is designed for shared file systems and supports protocols like NFS and CIFS. It is an excellent alternative if you require structured storage that multiple servers can access concurrently.
• Advantages: Provides structured, shared storage with centralized management; integrates well into OpenStack environments.
• Disadvantages: Requires an OpenStack infrastructure; may involve complex setup and management.
• Recommended Use: Best for large-scale applications that require structured, concurrent access to shared file systems.
• Ceph
• Description: A unified distributed storage system that offers block, file, and object storage.
• Advantages: Highly scalable and fault-tolerant; provides automatic data redundancy and high performance.
• Disadvantages: Complex setup and maintenance; higher infrastructure costs.
• Recommended Use: Ideal for large organizations needing high-performance, scalable video storage.
• Amazon S3 (Cloud Storage)
• Description: A robust cloud-based object storage solution with global accessibility.
• Advantages: Extremely scalable and secure; easily integrates with cloud applications; supports rapid file delivery via CDN.
• Disadvantages: Costs can vary based on usage; external dependency may add latency.
• Recommended Use: Ideal for organizations using AWS or seeking flexible cloud storage.
• GlusterFS
• Description: A distributed file system that creates a scalable storage volume across multiple servers.
• Advantages: Offers high scalability and good performance for distributed file access.
• Disadvantages: More complex to set up and manage; requires significant resources.
• Recommended Use: Suitable for large, distributed storage environments.

For scalability and fast access to video files across multiple servers, OpenStack Manila (for structured shared file storage) and Ceph are among the most robust solutions for large and complex environments. For simpler setups, NFS and CIFS/SMB might suffice, while Amazon S3 is ideal for cloud storage if you’re already using AWS.

Mounting a Shared Folder to /opt/una/storage

Before creating a new mount point, it is essential to back up the existing /opt/una/storage folder. This ensures that no data is lost during the transition. This process should be performed on both the main server and any video transcoding servers.

General Steps (Applicable for Both NFS and CIFS/SMB):

1. Backup the Existing Folder:
2. Move the current storage folder to a backup location:

cd /opt/una mv storage storage_bak 

1. Create a New Mount Point:
2. Create a fresh directory for the new shared storage:

sudo mkdir -p /opt/una/storage 

1. Mount the Shared Folder:
2. Use the appropriate command based on your protocol:

• For NFS:

sudo mount -t nfs <NAS_IP>:/path/to/shared_folder /opt/una/storage 

• Add the following line to /etc/fstab to mount automatically at boot:

<NAS_IP>:/path/to/shared_folder /opt/una/storage nfs defaults 0 0 

• For CIFS/SMB:

sudo mount -t cifs //<NAS_IP>/<Share_Name> /opt/una/storage -o username=<Username>,password=<Password>,uid=$(id -u),gid=$(id -g) 

• And add this line to /etc/fstab:

//<NAS_IP>/<Share_Name> /opt/una/storage cifs username=<Username>,password=<Password>,uid=$(id -u),gid=$(id -g) 0 0 

1. Restore Backup Files:
2. After the new shared folder is mounted, copy the files from the backup directory back into /opt/una/storage:

sudo rsync -av /opt/una/storage_bak/ /opt/una/storage/ 

1. Verify that all files have been transferred correctly (using, for example, diff -r). Once verified, you may optionally remove the backup:

sudo rm -rf /opt/una/storage_bak 

Required Modifications on UNA Servers

Both the main UNA server and the video transcoding servers (e.g., video1.example.com, video2.example.com, etc.) are nearly identical clones. However, you must make a small modification in the header.inc.php file on each server to differentiate them from the main site. This adjustment is crucial for proper configuration of each instance. For more details, please refer to the Video Transcoder Wiki.

Network Isolation for Video Transcoding Servers

For enhanced security and performance, video transcoding servers should run exclusively within the private network and must not be directly accessible from the public internet. This isolation ensures that these servers are dedicated solely to processing tasks and are shielded from potential external threats, while the main UNA server remains publicly accessible.

Storage Solutions for Containerized Environments

For complex, containerized environments (using Docker Swarm or Kubernetes), consider these advanced storage solutions:

• Ceph with Rook: Highly scalable, fault-tolerant storage with native Kubernetes integration.
• GlusterFS: Distributed file system that supports persistent volumes across clusters.
• Portworx: Cloud-native storage solution optimized for container orchestration.
• Amazon EFS: Fully managed, scalable storage ideal for AWS-based container environments.

Final Note on Database Connectivity

In this architecture, all video transcoding servers and the main UNA server connect to the same centralized MySQL cluster. This ensures that metadata, user information, and other critical data remain consistent across the entire system. To connect to the database edit the file header.inc.php file, all servers must contain this file in /opt/una/inc/header.inc.php

This architecture optimizes resource utilization by separating storage, processing, and distribution tasks while ensuring scalability and high availability through redundancy. The process involves first backing up the existing /opt/una/storage folder, then creating a new mount point, mounting the shared storage, and finally restoring your files. Additionally, modifications in the header.inc.php file on both the main and video transcoding servers are necessary for proper configuration. Video transcoding servers should operate exclusively within the private network for security reasons, and all servers connect to the same centralized MySQL cluster, ensuring consistent data across the system.

Make a full backup of your files and database before starting this process.

Happy coding!