Storing permanently locally doesn’t sound like a good solution…

If you could adjust the length of time to keep cached/proxy’d images locally and increase it significantly, I’d think that would be the preferred solution.

Not a lemm.ee user, but here’s my thoughts on #2 since it affects me via federation:

I am not a fan of how Lemmy chose to implement image proxying Specifically, federating the proxied URL.

That frequently prevents my instance from fetching a thumbnail locally (option 1 above). Which, ironically, increases the load on your server as my instance has to fetch it from your proxy every time instead of just once to generate a local copy here.

From a UI development standpoint, the proxied thumbnail URLs also make it harder to detect the image type (gif, static image, video) to handle rendering. It also complicates other proxying/caching methods I have in place. Ultimately, in the UI I develop, I’ve had to resort to passing thumbnail images through a function to un-proxy them so they can be handled sanely.

So I generally wish that admins avoid Lemmy’s proxying until it no longer federates the proxied URL and does something sane like just return that for the local API calls.

I would prefer the option which allows lemm.ee to run in the most sustainable manner

No opinion for the moment, but thank you for the very detailed post

I’m on a lot, and I scroll from All/Hot. I rarely see broken images. They do pop up, but not enough to ever bother me. The only option I’d avoid is method 1, because of that image debacle a few months ago. Regarding methods 2 and 3, they both seemed to work fine. I leave it to smarter minds than myself.

Good work on next, btw. Enjoy your weekend and thank you for everything you do!

Option 3 is the only one that seems sustainable long term. Donations will NEVER keep up with user growth, thus storage costs will balloon out of control.

Completely avoiding any chance of illegal content touching the servers should immediately have everyone agreeing on this option. I doubt anyone here is willing to foot legal bills and as such even minor legal actions would be the end of this instance.

Privacy is nice but ip logging is the simplest form to “protect” against with even a free VPN. If those claiming privacy concerns here aren’t already using a VPN and are depending purely on lemme.ee’s proxy then their internet hygiene needs an update.

As for usability, the image being deleted from external provider presents the same issue to the user between option 2 and 3. The cache from option 2 will inventually get cleared and it’ll fail to pull a fresh copy if deleted from the external hosts.

I say option 3. Sure, it’s annoying, but that’s our problem. Your problem is keeping the server operational, safe, and low-cost.

Considering the vote:content ratio, it looks like most users spend most of their time spectating. The spectators feed our egos and determine the trends in content by singular positive or negative votes. The spectator experience seems far more important to me and it should be the onus of the contributors to ensure their own privacy, just like they do in avoiding doxxing themselves via text.

Option 2 certainly follows in the vein of improved performance, but if real-world implementation is proving too unstable or creating too much overhead, then I say “fuck it, option 3 sounds great.”

Thanks for your hard work as always.

I’m in favor of moving away from proxying. Too many images break and proxying in general is very wasteful, having to download images from potentially small servers constantly would definitely get you ratelimited.

Passing through external images is OK. Many people often post external links anyways to sites like imgur and catbox because of the file size limits anyways.

I think the end goal would always to store images locally though - or at least caching them for extended periods of time. Don’t large instances like Lemmy World and huge Mastodon instances work this way? How do they manage the risk?

2+3: Try to fetch image, if you get it proxy it, if your storage gets full use LRU eviction, once evicted or some amount of time has passed delete it and don’t fetch again, ever. Fall back to pure 3 if there’s ever any issues with anything, including you not particularly feeling like implementing smart caching: Our referrer privacy is not your responsibility.

I would pick image pass through because I’m not necessarily concerned about the image hoster logging my IP. I have been more frustrated with broken links, so I am very much opposed to the current method.

Option 3.

I believe just passing through external images is the way to go, it’s always the one I opt for if I can

Hosting those images is gonna get expensive and that kinda sucks for a donation run platform when that money could be far better spent elsewhere

I think also using external image sources is more in line with the idea of decentralisation, Lemmy isn’t an image host it’s a link aggregator and forum - I believe most image hosting sites will be far better at loading images quickly than Lemmy’s implementation could ever be

Definitely OK with either option 2 or 3. And I trust @sunaurus to choose what’s best for this instance.

I think it’s important for us to be mindful of content retention for posterity’s sake if we want Lemmy to compete with Reddit long-term. If possible, I’d hope we can avoid dead image links like we see with old forums and photobucket pics, for example.

Wow, thanks for the full transparency. You are awesome!

My opinion would be option 2 (proxy requests) , but with a higher cache TTL or simple a LRU (Least Recently Used) Cache.

If you’re getting throttled, it could be mitigated by increasing the cache retention period (or improving the cache hits).

Another improvement : Would it be possible to change the proxy, so that if the proxied requests are throttled, it simply sends the user a http-302 to the origin (instead of a broken image)?

Regarding option 1 (full cache) : I greatly appreciate your desire to hide/protect your users ip, but it is outside the scope of what I expect from a Lemmy server. Maybe you could market and upsell this increased privacy as a subscription based feature. However, if I want privacy - I’ll use a VPN.

Regarding option 3 (User fetches content from origin) : From a users perspective, I really don’t want my Lemmy experience to be based on hitting a bunch of (potentially) unreliable services. When I, as a lemm.ee User, request a post from Lemmy.world (for example), lemm.ee will proxy and cache that post and the comments. This is the distributed nature of Lemmy (as far as I understand). Why restrict this caching to just posts/threads/comments and not include images (which, let’s face it, are as meaningful as pure text - especially wrt memes).