Thanks!
Cost is not the bottleneck in this case... The problem is that I am rather ignorant about the options and their benefits/limitations. Moving the images the first time was painfully slow because of those same rate limits, and I expect the next migration to be the same, so I want to make a better choice next time and would rather find a solution with the current provider 😅
Thanks for the heads up. I am still trying to resolve this without a migration... I will try again to get a response from them as they have not replied in a week.
This is the current status:
The instance is serving the images via object storage. Specifically, I am making use of Contabo to save and serve the images.
I now know that the default limits are 250 requests / second and 80 Mbit/s: https://help.contabo.com/en/support/solutions/articles/103000275478-what-limits-are-there-on-object-storage-
It appears to me like when the requests are exceeded, the "Too many requests" error is triggered and it takes a few seconds before the requests are accepted again. This can happen if few users access the front page at once as this will fetch all of the thumbnails and icons on the page.
I have been in touch with Contabo's customer support via e-mail. But they mis-understood my original e-mails and thought I was speaking about increasing the maximum number of images that can be stored (3 million by default). I have clarified that I want to increase the rate limit and have been waiting for their response for a few days now.
The other solution would be to move the images to a different object storage provider. The migration is also limited to the 250 requests/s and 80 Mbit/s, so it will require turning off the images for 4 - 7 days while all the images are moved... Since I am not familiar with the policies of other object storage providers I would also need to do research to avoid falling into the same trap.
So, I am hoping that Contabo's support will get back to me soon and allow me to increase the rate limits, as this would be the most straight forward approach.
And you are doing a great job at that! 😄
Very interesting article, thanks for sharing. I agree that it is a good one to pin!!
I have been reaching out to the object storage provider to see if I can increase the rate limits... Unfortunately I might need to change to a different provider to overcome this. Since the migration takes several days, especially so because of those same rate limits, I would rather avoid this...
That's an error I had not seen before, but I also just encountered with this specific post. I will investigate, thanks.
This error is a rate limit from the object storage provider. I did not know of this limit when I chose them, and I still have not found a way to change the limit. I will send them an e-mail. If the limit can't be increased, one option is to pick another object storage provider, but the migration takes days.
Check in your settings whether you have disabled the visibility of bot responses. This can happen if bots replied to you and your settings are set to not see them.
Thank you for being alert! I have banned them instance-wide now.
Inderdaad, ik ben ook voor die marketing gevallen.
Denk je dat mmWave binnenkort beschikbaar zal zijn? Op dit moment heb ik een sub-6 GHz 5G-router voor internet thuis. Ik ben benieuwd hoe een mmWave-router zou presteren als er een zendmast in de buurt is.
Publishing in a more prestigious journal usually means that your work will be read by a greater number of people. The journal that a paper is published on carries weight on the CV, and it is a relevant parameter for committees reviewing a grant applicant or when evaluating an academic job applicant.
Someone who is able to fund their own research can get away with publishing to a forum, or to some of the Arxivs without submitting to a journal. But an academic that relies on grants and benefits from collaborations is much more likely to succeed in academia if they publish in academic journals. It is not necessarily that academics want to rely on publishers, but it is often a case of either you accept and adapt to the system or you don't thrive in it.
It would be great to find an alternative that cuts the middle man altogether. It is not a simple matter to get researchers to contribute their high-quality work to a zero-prestige experimental system, nor is it be easy to establish a robust community-driven peer-review system that provides a filtering capacity similar to that of prestigious journals. I do hope some alternative system manages to get traction in the coming years.
Cross-posting to the OpenSource community as I think this topic will also be of interest here.
This is an analysis of how "open" different open source AI systems are. I am also posting the two figures from the paper that summarize this information below.
ABSTRACT
The past year has seen a steep rise in generative AI systems that claim to be open. But how open are they really? The question of what counts as open source in generative AI is poised to take on particular importance in light of the upcoming EU AI Act that regulates open source systems differently, creating an urgent need for practical openness assessment. Here we use an evidence-based framework that distinguishes 14 dimensions of openness, from training datasets to scientific and technical documentation and from licensing to access methods. Surveying over 45 generative AI systems (both text and text-to-image), we find that while the term open source is widely used, many models are ‘open weight’ at best and many providers seek to evade scientific, legal and regulatory scrutiny by withholding information on training and fine-tuning data. We argue that openness in generative AI is necessarily composite (consisting of multiple elements) and gradient (coming in degrees), and point out the risk of relying on single features like access or licensing to declare models open or not. Evidence-based openness assessment can help foster a generative AI landscape in which models can be effectively regulated, model providers can be held accountable, scientists can scrutinise generative AI, and end users can make informed decisions.
Figure 2 (click to enlarge): Openness of 40 text generators described as open, with OpenAI’s ChatGPT (bottom) as closed reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points. RL may refer to RLHF or other forms of fine-tuning aimed at fostering instruction-following behaviour. For the latest updates see: https://opening-up-chatgpt.github.io
Figure 3 (click to enlarge): Overview of 6 text-to-image systems described as open, with OpenAI's DALL-E as a reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points.
There is also a related Nature news article: Not all ‘open source’ AI models are actually open: here’s a ranking
PDF Link: https://dl.acm.org/doi/pdf/10.1145/3630106.3659005
ABSTRACT
The past year has seen a steep rise in generative AI systems that claim to be open. But how open are they really? The question of what counts as open source in generative AI is poised to take on particular importance in light of the upcoming EU AI Act that regulates open source systems differently, creating an urgent need for practical openness assessment. Here we use an evidence-based framework that distinguishes 14 dimensions of openness, from training datasets to scientific and technical documentation and from licensing to access methods. Surveying over 45 generative AI systems (both text and text-to-image), we find that while the term open source is widely used, many models are ‘open weight’ at best and many providers seek to evade scientific, legal and regulatory scrutiny by withholding information on training and fine-tuning data. We argue that openness in generative AI is necessarily composite (consisting of multiple elements) and gradient (coming in degrees), and point out the risk of relying on single features like access or licensing to declare models open or not. Evidence-based openness assessment can help foster a generative AI landscape in which models can be effectively regulated, model providers can be held accountable, scientists can scrutinise generative AI, and end users can make informed decisions.
Figure 2 (click to enlarge): Openness of 40 text generators described as open, with OpenAI’s ChatGPT (bottom) as closed reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points. RL may refer to RLHF or other forms of fine-tuning aimed at fostering instruction-following behaviour. For the latest updates see: https://opening-up-chatgpt.github.io
Figure 3 (click to enlarge): Overview of 6 text-to-image systems described as open, with OpenAI's DALL-E as a reference point. Every cell records a three-level openness judgement (✓ open, ∼ partial or ✗ closed). The table is sorted by cumulative openness, where ✓ is 1, ∼ is 0.5 and ✗ is 0 points.
There is also a related Nature news article: Not all ‘open source’ AI models are actually open: here’s a ranking
PDF Link: https://dl.acm.org/doi/pdf/10.1145/3630106.3659005
We are having a pumpking growing competition at work and I live in an apartment, so I'm working with what I have 😆
The plant already produced many male flowers. From what I have read, the male flowers usually come out 10 - 14 days before the female flowers. They open up for a single day and then they close and fall off.
I found out that tey are edible, so I stuffed a few of them with some left overs as a culinary experiment.
And the first female flower has arrived!
Good news! Just got a reply from them and they have increased the connection limit. They did not specify what the new number is, but hopefully it is high enough to not be an issue for the foreseeable future.
So, if you do run into other similar reports after this comment I would appreciate it if you tag me again.