Profile and cover pictures seem to expire on S3 cloud bucket
Hi folks,
I have a problem with my remote storage. It seems that profile and cover pictures of persons and organizations expire after a while and the links to my storage bucket are invalid. I'm not sure if that issue is on UNA's side or on my storage provider. Does anybody have the same issue?
The link is for example https://s3-eu-central-1.ionoscloud.com/busily/bx_persons_pictures_resized/i/ik/ikd/ikdm8pxd6e5ifylhc63bkfztwjsmqi5v.jpg. But it has been automatically deleted.
-
By the way, what are these automatically generated links to my post for, @Leonid S ?
-
Hello @thomlin !
Could you please specify the situation with your storage account? Why do we see the "no such key" message in the given response?
-
Because it was deleted for an unknown reason. I try to find out if that's related to the UNA framework or to my storage provider. Following the path of the example pic, it turns out that the whole folder structure below bx_perons_pictures_resized has vannished.
My guess is, that data that is not called in a certain time-frame gets automatically deleted. For that reason, I suspect my cloud storage provider to have some rules active that cause that automatic deletion. Just want to make sure that it cannot be on UNA framework's side, Leonid.
-
But if the problem is on the UNA framwork's side, it probably has something to do with the caching engine.
-
Dear @Leonid S , I forwarded the support message of my S3 cloud storage host by email. It seems to be a problem with the caching mechanism.
-
I'm losing data every day. Seems to have to do with an expiration timestamp of 90 days. @Leonid S , please help!
My admin accont just lost its cover and its profile pictures.
-
Next resized picture has been deleted by my UNA installation, @Alex T⚜️ . The profile picture is only still available because browser has cached it, I guess.
-
Could you please provide access to your site Studio via private message ?
-
UNA has mechanism to delete resized images if they were not accessed for certain period of time. However when image is requested again, then it should be autogenerated from the original image. It works fine on many websites including una.io
You can disable this pruning mechanism by executing the following query and then clear DB cache:
UPDATE `sys_objects_transcoder` SET `atime_tracking` = 0, `atime_pruning` = 0However it's better to figure out what is the problem, so deleted images can be regenerated again. You can try to check logs/sys_transcoder.log for any any errors related to the transcoder.
Also you have outdated UNA version, it's strongly recommended to update to the latest one - https://una.io/wiki/Upgrade
-
Ok, will check the log file. Thanks for investigation. Since that is my production site, I did not update to the first stable 13 release, yet. Will do that after 13.0.1 has been released and finally tested.
-
I just had the same problem. I also use IONOS test cloud. I installed a new installation and made a new bucket for it on IONOS. Images wouldn't upload. However when i use the first bucket i created it was possible to upload the logo and profile/album images. Maybe it's because of the Test mode. I guess i'll will find out when i activate my account.
Beside this IONOS is much easier to setup than amazon/wasabi in my experience.
- create bucket
- write down the key and secret key
- choose your region e.g.
s3-eu-south-2.ionoscloud.comdon't use the ` when you place this in Studio/settings > endpoint.
-
Do you already have a update for this Thomlin? As i have similar experience.
-
@Alex T⚜️ answered to my support request and gave me some hints on the error messages in my sys_transcoder.log file. I assumed that in my case deletion of files in the S3 bucket occurs because I have two UNA framework instances running on the same server. One in the root folder of my server and one in a subfolder (this was recommended long time ago to have the option to use premium modules with one single UNA license), which is my test environment before going live with new modules. Alex confirmed my assumption that two instances have to have two seperate buckets. After that I added a new bucket for my test server and since that I'm observing.
Alex told me further that resized pictures expire after 30 days. But when they are called up again, they should be created on the fly from the original uploaded pictures. But in my case it seems that I've lost some of those original pictures, maybe caused by the shared single bucket for two UNA instances.
Alex told me, we'll wait until other UNA framework users report similar behavior. Well, @Alex T⚜️ , at least another case.
The S3 buckets on the IONOS cloud servers seem to operate without issues. Their support staff could even tell me how many object deletion instructions were sent over the UNA framework API. The S3 buckets are not test versions, as far as I know. They are ready for production. IONOS once charged me. I think it was 3 cents for a month. After that they stopped charging, because I think expenses for charging exceeded profits by far. :-)
-
Thanks for your comprehensive answer. In fact, it should also be possible during the test period to run with 2 installations on a server (shared hosting), where you have to use 2 separate buckets. I'm will test it again.
