Hello all,
I am trying to understand what I need to do to ensure that my Lemmy instance can be federated, found, and usable with other instances and users. https://geekroom.tech/ is my instance.
Currently, the issues I’m running into are:
-
If I search for my own user@myInstance.tld from any other instance, nothing comes up.
-
When I try to add my instance to FediDB.org, it shows that my “Instance is invalid, unreachable or unavailable.”
-
When I subscribe to other communities from my instance, I am seeing posts, but no comments appear at all. I have verified that those same threads appear on the instance that they are generated on. (My understanding is that I should be able to see these comments and be able to respond to them)
-
All of the other communities that I have attempted to subscribe to are showing “Subscribe Pending”, but the 2 communities that I’ve created on my instance are obviously showing as Joined.
-
If I comment on a post from within my own instance, that post does not show up on the original post. (again, may be my misunderstanding of how to use Lemmy lol)
I know I’m asking a lot, but it seems to me that obviously something is misconfigured, or I’m using Lemmy wrong.
What am I doing wrong? I can post configs if needed.-
Edit1: Looks like, based on some changes I made here: https://lemmy.world/comment/15359241, that federation is now partially working as it is sending out activities… however, after waiting some more time eventually it shows that there are more activities that are behind (possibly after trying to edit a comment that I made and was trying to get it to post something to the feed)
Edit2: I have attempted to unsubscribe and resubscribe and it appears that is not working even after 24 hrs. The logs are showing when I am attempting to subscribe, and then the logs show that I am 1 activities behind for that action on all the subscribed servers. In addition, I changed the following:
set $proxpass "http://127.0.0.1:10633/";
to
set $proxpass "http://127.0.0.1:1234/";
However, I’m still not seeing any action on being up to date. I’m not seeing any new posts from other communities and I’m not seeing my comments on those remote subscribed communities showing up on the original posts.
Edit3: After doing some more digging, I believe I have found another part of the issue. So there’s the NGINX reverse proxy that I have installed at the host level, which redirects to port 127.0.0.1:10633 for the geekroomtech-proxy-1 container to serve content from.
If I set the initial $proxpass value to port 1234 it gives me a 502 bad gateway when doing the curl tests. If I set it as: set $proxpass “http://127.0.0.1:10633/”; then it returns values properly.
However, the issue becomes when I have these next two conditional sections that breaks the curl tests and goes back to 502 bad gateway:
if ($http_accept = "application/activity+json") {
set $proxpass "http://127.0.0.1:8536/";
}
if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") {
set $proxpass "http://127.0.0.1:8536/";
}
When inside the internal proxy that lemmy provides, I can issue: curl -H “Accept: application/activity+json” http://lemmy:8536/c/rDataHoarder and returns information properly.
So… I exposed the ports 8536 on the Lemmy container, and magically, I’m no longer getting the 502 bad gateway errors.
ports:
- "127.0.0.1:8536:8536"
While I’m not seeing any failures for activity trying to come in, I’m still seeing communities aren’t updating, subscribes are still pending, and I’m still not getting any activity showing up in the feel.
Did you just set it up? It took my instance a few days to get federating. Also, sometimes you need to leave and resub a com.
Yes, set it up today. Was having issues with sending emails only to find out I had to just specify a different “hostname” in the lemmy.hjson file other than GeekRoom.Tech… so www.geekroom.tech got that working.
I enabled the lemmy_federation trace settings and am seeing that it is finally federating and trying to send updates. Being that no one but myself is on it currently, I’m not terribly surprised by the instances being “behind”.
Example:
2025-02-28T00:32:14.394971Z DEBUG lemmy_federate::stats: lemmy.nz: Ok. 1 activities behind 2025-02-28T00:32:14.394974Z DEBUG lemmy_federate::stats: lemmy.blahaj.zone: Ok. 1 activities behind 2025-02-28T00:32:14.394976Z DEBUG lemmy_federate::stats: lemmy.cafe: Ok. 1 activities behind 2025-02-28T00:32:14.394979Z DEBUG lemmy_federate::stats: libranet.de: Ok. 1 activities behind 2025-02-28T00:32:14.394982Z DEBUG lemmy_federate::stats: theprancingpony.in: Ok. 1 activities behind 2025-02-28T00:32:14.394984Z DEBUG lemmy_federate::stats: lemmy.cloudhub.social: Ok. 1 activities behind 2025-02-28T00:32:14.394987Z DEBUG lemmy_federate::stats: lemmy.dbzer0.com: Ok. 1 activities behind 2025-02-28T00:32:14.394990Z DEBUG lemmy_federate::stats: monero.town: Ok. 1 activities behind 2025-02-28T00:32:14.394992Z DEBUG lemmy_federate::stats: lemmy.hybridsarcasm.xyz: Ok. 1 activities behind 2025-02-28T00:32:14.394995Z DEBUG lemmy_federate::stats: 1337lemmy.com: Ok. 1 activities behind 2025-02-28T00:32:14.394998Z DEBUG lemmy_federate::stats: palaver.p3x.de: Ok. 1 activities behind 2025-02-28T00:32:14.395001Z DEBUG lemmy_federate::stats: jlai.lu: Ok. 1 activities behind 2025-02-28T00:32:14.395003Z DEBUG lemmy_federate::stats: lemmy.toldi.eu: Ok. 1 activities behind 2025-02-28T00:32:14.395006Z DEBUG lemmy_federate::stats: discuss.online: Ok. 1 activities behind 2025-02-28T00:32:14.395013Z DEBUG lemmy_federate::stats: lemmy.crimedad.work: Ok. 1 activities behind 2025-02-28T00:32:14.395016Z DEBUG lemmy_federate::stats: mander.xyz: Ok. 1 activities behind 2025-02-28T00:32:14.395018Z DEBUG lemmy_federate::stats: programming.dev: Ok. 1 activities behind 2025-02-28T00:32:14.395021Z DEBUG lemmy_federate::stats: fedia.io: Ok. 1 activities behind 2025-02-28T00:32:14.395024Z DEBUG lemmy_federate::stats: pawb.social: Ok. 1 activities behind 2025-02-28T00:32:14.395027Z DEBUG lemmy_federate::stats: feddit.it: Ok. 1 activities behind 2025-02-28T00:32:14.395030Z DEBUG lemmy_federate::stats: lemmy.blackeco.com: Ok. 1 activities behind 2025-02-28T00:32:14.395032Z DEBUG lemmy_federate::stats: www.foxyhole.io: Ok. 1 activities behind 2025-02-28T00:32:14.395035Z DEBUG lemmy_federate::stats: lemmy.world: Ok. 1 activities behind 2025-02-28T00:32:14.395038Z DEBUG lemmy_federate::stats: sopuli.xyz: Ok. 1 activities behind 2025-02-28T00:32:14.395041Z DEBUG lemmy_federate::stats: lemmy.nowsci.com: Ok. 1 activities behind 2025-02-28T00:32:14.395043Z DEBUG lemmy_federate::stats: lemmy.thewooskeys.com: Ok. 1 activities behind 2025-02-28T00:32:14.395046Z DEBUG lemmy_federate::stats: startrek.website: Ok. 1 activities behind 2025-02-28T00:32:14.395058Z DEBUG lemmy_federate::stats: lemmy.zip: Ok. 1 activities behind 2025-02-28T00:32:14.395061Z DEBUG lemmy_federate::stats: hackertalks.com: Ok. 1 activities behind 2025-02-28T00:32:14.395064Z DEBUG lemmy_federate::stats: feddit.online: Ok. 1 activities behind 2025-02-28T00:32:14.395067Z DEBUG lemmy_federate::stats: midwest.social: Ok. 1 activities behind 2025-02-28T00:32:14.395070Z DEBUG lemmy_federate::stats: kbin.earth: Ok. 1 activities behind 2025-02-28T00:32:14.395072Z DEBUG lemmy_federate::stats: feddit.rocks: Ok. 1 activities behind 2025-02-28T00:32:14.395076Z INFO lemmy_federate::stats: 0 others up to date. 61 instances behind. 2025-02-28T00:32:14.662688Z INFO lemmy_federate: Federating to 61/61 instances (0 dead, 0 disallowed)
Can you just tell us what is your domain, so we can help you troubleshoot it?
https://geekroom.tech/ is my instance.
Seems to be working fine now. My instance can find yours without any problems.
can your instance still see mine and post to it? Asking because I haven’t gotten any new posts in 24 hours when I know those same remote communities have newer content than that. Additionally, I can’t see any comments that have previously been made…and my comments still don’t sync and show up in the original posts feed.
hmm… ok. I’m guessing it’s maybe the instances that i’ve tried searching on that aren’t picking it up yet or something. Because here on lemmy.world and also sh.itjust.works neither are picking up my instance.
What is your instance? It sounds like your instance is unreachable or otherwise not federating right, when others try to reach it.
If you DM or comment with what your instance is, I can do some checking on whether your instance is accepting federation traffic as it should be.
https://geekroom.tech/ is my instance.
Definitely seems like it works now. I can see your communities, and I did a test post and it went to your instance (then I deleted it).
I would tend to guess that it’s the Cloudflare config that was the issue.
can your instance still see mine and post to it? Asking because I haven’t gotten any new posts in 24 hours when I know those same remote communities have newer content than that. Additionally, I can’t see any comments that have previously been made…and my comments still don’t sync and show up in the original posts feed.
Hm, I saw no posts on your instance for !rDataHoarder@geekroom.tech, so I tried to make one on my instance to see if it would show up, and now your server’s giving me an internal error viewing that community. Do you know what’s up with that?
I think the time period during which whatever-it-was was making things not work, may have left your federation of communities in a borked state. One thing you can try: Unsubscribe from some particular remote community from all users on your instance (so your instance will request to stop getting updates). Then, resubscribe, and see if you start getting updates again. The initial request to get updates might have gotten swallowed by the brokenness.
This might be a good question for the devs, also. Both because they can help more and because they should know about it as an issue (whether or not it was caused by some Cloudflare thing and whether or not it was your “fault” in setting it up, both of those questions I have no idea.)
There aren’t any posts on any communities on my instance yet which might be an issue, sure.
What error are you getting when trying to post?
I do see that there is a post from you around half an hour ago. Something that is odd to me is that rDataHoarder shows rDataHoarder@geekroom.tech where as rhomelab just shows rhomelab lol
I’m guessing that it’s because a federated post was made to it? A local post made to it didn’t change how it shows in rhomelab
It says: The server returned this error:
couldnt_find_community. This may be useful for admins and developers to diagnose and fix the error.rDataHoarder also doesn’t show in the list of local communities.
Did you change the domain of your host, after it had already done some federation and created databases, or something like that? I do see the rDataHoarder community in the list of non-local communities, which is a real bad type of error to be seeing. You didn’t do any hacking around in the database to fix things or anything did you? That should never be that way as far as I know.
Originally, I had set the “hostname” in lemmy.hjson as just “geekroom.tech” but that was causing issues with trying to send email. So I changed it to the hostname of the server prd-pub-pod11.geekroom.tech" … then found out that when it sends emails, it uses that hostname as where to send people to in order to verify and login… no bueno.
So then I changed it to www.geekroom.tech and now just have a redirect at cloudflare for www to naked domain
It sounds like I need to reinit the entire install …
It appears that adding the below code has seemingly resolved the not receiving inbox messages part, but has not resolved anything else.
First, I disabled the Cloudflare proxying because I saw the troubleshooting guide mentions that Cloudflare proxy may cause issues.
Second, added the following:
# Attempting to resolve federation set $proxpass "http://127.0.0.1:10633/:"; if ($http_accept = "application/activity+json") { set $proxpass "http://127.0.0.1:8536/"; } if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") { set $proxpass "http://127.0.0.1:8536/";You should put the lemmy-ui’s port (8635) first then in the conditions blocks, the lemmy’s port
set $proxpass "http://0.0.0.0/:{{ lemmy_ui_port }}"; if ($http_accept = "application/activity+json") { set $proxpass "http://0.0.0.0/:{{ lemmy_port }}"; } if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") { set $proxpass "http://0.0.0.0/:{{ lemmy_port }}"; } proxy_pass $proxpass;https://join-lemmy.org/docs/administration/troubleshooting.html
So when I look at the docker-compose file, the only exposures are 10633 on nginx-proxy which routes to 8536. Docker ps -a is showing lemmy-ui as 1234/tcp and lemmy as 8536.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 530dd3852e24 nginx:1-alpine "/docker-entrypoint.…" 15 hours ago Up 15 hours 80/tcp, 127.0.0.1:10633->8536/tcp geekroomtech-proxy-1 e5976396c4f2 dessalines/lemmy-ui:0.19.9 "docker-entrypoint.s…" 15 hours ago Up 15 hours (healthy) 1234/tcp geekroomtech-lemmy-ui-1 a91f86523a95 dessalines/lemmy:0.19.9 "lemmy_server" 15 hours ago Up 15 hours 8536/tcp geekroomtech-lemmy-1 15c8a65b2e02 asonix/pictrs:0.5.16 "/sbin/tini -- /usr/…" 15 hours ago Up 15 hours 6669/tcp, 8080/tcp geekroomtech-pictrs-1 135f75ceccf4 pgautoupgrade/pgautoupgrade:17-alpine "/usr/local/bin/dock…" 15 hours ago Up 15 hours (healthy) 5432/tcp geekroomtech-postgres-1So you’re saying it should look like this instead?
set $proxpass "http://0.0.0.0/:1234"; if ($http_accept = "application/activity+json") { set $proxpass "http://0.0.0.0/:8536"; } if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") { set $proxpass "http://0.0.0.0/:8536"; } proxy_pass $proxpass;is that right?
yes 😅,
For me thats working like this anyway, ^^I had to set it as this because if I used port 1234 for lemmy-ui then it would give me 502 bad gateway errors.
set $proxpass "http://127.0.0.1:10633/"; if ($http_accept = "application/activity+json") { set $proxpass "http://127.0.0.1:8536/"; } if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") { set $proxpass "http://127.0.0.1:8536/"; }But this works.
