A posthumous sequel to 1984? Sounds a bit dystopian to me!

Then again I enjoyed Grendel, a retelling of Beowulf from a different point-of-view, so perhaps Julia could be enjoyed in the same way.

Agreed. My copy lost this documentation link in the original which gives more detail about the horizontal scaling: https://join-lemmy.org/docs/administration/horizontal_scaling.html.

It seems really straightforward (which is a good thing), each backend Lemmy_server handles incoming requests and also pulls from a shared queue of other federation work.

Time zones are an endless source of frustration, this one doesn’t sound too bad though:

Going forward, all timestamps in the API are switching from timestamps without time zone (2023-09-27T12:29:59.113132) to ISO8601 timestamps (e.g. 2023-10-29T15:10:51.557399+01:00 or Z suffix). In order to be compatible with both 0.18 and 0.19, parse the timestamp as ISO8601 and add a Z suffix if it fails (for older versions).

https://github.com/LemmyNet/lemmy/pull/3496

This data structure uses a 2-dimensional array to store data, documented in this scala implementation: https://github.com/twitter/algebird/blob/develop/algebird-core/src/main/scala/com/twitter/algebird/CountMinSketch.scala. I’m still trying to understand it as well.

Similar to your idea, I had thought that by using k bloom filters, each with their own hash function and bit array, one could store an approximate count up to k for each key, which also might be wasteful or a naïve solution.

PDF link: http://www.eecs.harvard.edu/~michaelm/CS222/countmin.pdf

I haven’t used them in Spark directly but here’s how they are used for computing sparse joins in a similar data processing framework:

Let’s say you want to join some data “tables” A and B. When B has many more unique keys than are present in A, computing “A inner join B” would require lots of shuffling if B, including those extra keys.

Knowing this, you can add a step before the join to compute a bloom filter of the keys in A, then apply the filter to B. Now the join from A to B-filtered only considers relevant keys from B, hopefully now with much less total computation than the original join.

Collage sounds really interesting , will check it out. Another variation on bloom filter I recently learned about is count-min-sketch. It allows for storing/incrementing a count along with each key, and can answer “probably in set with count greater than _”, “definitely not in set”.

Thanks for adding more detail on the DB use-cases!

Well yes, but this is an internet forum for flashlights. That’s pretty weird too.

The reflector of the flashlight is built so light coming from a very small source (like the filament of an incandescent bulb) is directed forward in a focused beam.

I agree, but I also think that using a modern LED with a single source of forward-facing light is fine. However the emitter would need to be properly positioned in the light.

Here’s a very similar host (from one of the best low-cost flashlight makers) showing a properly aligned LED and reflector:

Product link: https://www.aliexpress.us/item/3256801849471618.html

I found a few references to this exact model on candlepowerforums.com which I believe has more folks who own(ed) incandescent lights. Not that has been such a long time, but LED technology advanced very quickly. Not sure if that will help your search.

There’s also a few people over on !flashlight@lemmy.world !

@Spicy3d@lemmy.world check out this diffuser!

Cool project and post! There’s also !flashlight@lemmy.world if you’d like to cross post.

gets hot enough to burn you if you leave it on for too long

That Fenix looks like a reliable light and is designed with temperature regulation but the limit might be pretty high and of course it is being used in an enclosure.

Although your current role wouldn’t seem very senior at a large organizational, “senior“ is a relative term and at this company it seems like you are the engineer with ownership responsibilities over the end-to-end software development of a production system. So it might still be reasonable to use a senior title if there are other benefits