New API endpoints launched, feedback needed

The_Moose

Good news everyone, today we've launched the alternative API endpoint cf-api.99stack.com as a decentralized load balancer for api.99stack.com. It's the same software, it's connected to the same database cluster but it's running on a different server. It also take advantage of Cloudflare CDN, now you could argue both for and against that. But the point is that we'll keep both endpoints active even in the future which means you can choose whichever best fits your needs and preferences in terms of privacy, security and loading speed.

Now in theory these two, as well as new endpoints added in the future will take advantage of peer to peer sync and thereby stay up to date. However, as history has taught us, theory and real life isn't always the same which is why we need your feedback regarding this update, let us know if something doesn't work as supposed or if you'll have any other questions or concerns regarding the setup.

The_Moose

Here's a very basic illustration of the current setup:

Database servers, peer-to-peer synced
|us-central|    <-->    |eu-west|    <-->    |eu-central|    <-->    |as-central|    <-->    |as-south|    --> Cold backups [eu-north]
          |                                 |                                 |
 -----------------------------[ Firewall ]------------------------------
          |                                 |                                 |
 -------------------------[ Loadbalancer ]--------------------------
          |                                 |                                 |
Web servers, peer-to-peer synced
|us-central|    <-->    |eu-west|    <-->    |eu-central|    --> Cold backups [eu-north]
          |                                 |                                 |
      cf-api                          api-a                           api-b
          |                                 |                                 |
   [ Cloudflare ]                   ----[ Loadbalancer ]----
          |                                                   |
   cf-api.99stack.com               api.99stack.com

Marco_Weiss

Nothing negative but the setup does seem a bit overkill, professional and secure but yet overkill. cf-api works very well as expected indeed. Tho some functions where a little bit slow, I suppose that has to do with latency seeing that the new site appears to be somewhere in North America. I'm in Germany BTW and I suppose eu-central is in Frankfurt which is why the old endpoint responds very fast for me 🙂

Kenneth_Crayton

It seems to be built for scaling, reliability and security which is great, sure some speed will be lost now in the short term but in the long term this is excellent. As an American I can confirm what Macro said, cf-api which is hosted on this side on the pond responds almost 200ms faster then your EU endpoints. Don't be shy now, please add an Asian endpoint for our friends over there so they can enjoy a faster API too.

Allan_Virtanen

I don't really like how Cloudflare is taking over the internet, but I do appreciate that you keep your own endpoints that doesn't involve Cloudflare. Good setup BTW. Assuming the loadbalancer picks the nearest server, first I guess that explains the improved response times. See there, it works brilliant, and Cloudflare is not involved. 👍️

Magnus_Lind

Where are these servers located? I was mainly thinking of eu-north which I haven't seen before.

The_Moose

Our primary servers are located as follows:

us-central:   Montreal, Canada
eu-west:      Dublin, Ireland
eu-central:   Frankfurt, Germany
as-central:   Mumbai, India
as-south:     Sydney, Australia

Each region contains multiple servers spread over different datacenters located within the region for improved redundancy and security. Now eu-north is a little special, unlike the primary regions which remains within one the borders of a larger city, this region spans across multiple countries in northern Europe, including Iceland, Finland, Norway and Denmark. Sweden is excluded from the setup.

George_Hegland

It was slow directly after the update, but now it runs fast again. 👍️

Alan_Rowell

Works great, it was shown as being down for a while yesterday but it didn't seem to be down and now it's faster than ever. Any ideas what's going on?

The_Moose

Alan_Rowell Works great, it was shown as being down for a while yesterday but it didn't seem to be down and now it's faster than ever. Any ideas what's going on?

Some miss configuration in Cloudflare dashboard, while adding more nodes to the new endpoint occurred. This caused a java script challenge to be activated once the script checking up-time attempted to ping the API. So even if it was reported as down it was never actually down. Only the up-time ping bot where blocked.

In fact, cf-api.99stack.com is now multi-regional. We're gonna run a few tests during next week where random host nodes are taken down so simulate server crashes, DDoS attacks and some more to see if all system stays online.