I don‘t know if this is the right place to ask, but is this an OK build for a first start in selfhosting? Is there anything obvious I could improve? I‘d mainly run Nextcloud, Immich, some Docker containers and smaller web applications and cronjobs.
I don‘t know if this is the right place to ask, but is this an OK build for a first start in selfhosting? Is there anything obvious I could improve? I‘d mainly run Nextcloud, Immich, some Docker containers and smaller web applications and cronjobs.
how are you using 128 GB? Genuinely curious because I run over 20 services with only 8 GB
All optimized for performance and fine tuned as well, eg. lets say you run mariadb vs how I run it:
innodb_buffer_pool_size = 8G innodb_flush_log_at_trx_commit = 2 innodb_log_file_size = 2G innodb_log_buffer_size = 32M innodb_max_dirty_pages_pct = 90 innodb_io_capacity=5000 innodb_io_capacity_max=20000 innodb_read_io_threads=8 innodb_write_io_threads=8 query_cache_type = 1 query_cache_limit = 2M query_cache_min_res_unit = 2k query_cache_size = 128M tmp_table_size= 128M max_heap_table_size= 128M [mysqld] max_connections = 200 character_set_server = utf8mb4 collation_server = utf8mb4_general_ci transaction_isolation = READ-COMMITTED binlog_format = ROW innodb_file_per_table=1 # Increase open files based limits.conf value open_files_limit=65535Same idea for Postgresql… You can run “postgres” or… actually run postgresql in production correctly like:
shared_buffers = 6GB work_mem = 20MB maintenance_work_mem = 2GB maintenance_io_concurrency = 200 max_worker_processes = 14 max_parallel_workers_per_gather = 4 max_parallel_maintenance_workers = 4 max_parallel_workers = 12 synchronous_commit = off commit_delay = 300 checkpoint_timeout = 30min max_wal_size = 60GB min_wal_size = 4GBActually now that I check your username I think I have seen your instances before, though it seems like your website is down at the moment. But thank you for service to open source !
nah its not down… maybe you use a VPN.
I do. Do you block all VPNs? Or is it just rate limiting?
I block some misbehaving data centers. Which are often also used by VPN providers. So it’s more collateral damage.
No I fully block some data centers not just rate limit.
Unfortunate but understandable. I believe there are public lists of the IPs of major VPN providers if you wanted to make an exception, but as VPNs are sometimes used for scraping too, I can see why you might not want that.
Yea so the same servers and IP addresses VPN provider use (which are again just cloud providers) are also used for other purposes mainly by scrapers and DDos attacks indeed. Too bad all those cloud providers do not act accordingly in the past 5 - 10 years. Since its getting worse and worse. With the increase of datacenters and centralization, I consider it a duty of these companies to take action to stop these scammers, spammers, scrapers, and attackers.
However, that is often not the case. Now I must say, I created Angie Guardian myself (alternative to Anubis). So hopefully soon I can slowly open some of ASN bans. And see how it goes.
Do you know if the scrapers/bots are using the VPN providers? Or if they are just using VPSes in the same datacenter as the VPN, and the datacenter is just NATing all egress traffic to have the same IPv4. In that case I wonder if IPv6 could help distinguish between VPN traffic and bot traffic…
Though ultimately a gate like Anubis or Angie Guardian might be the best solution here so nice work.