• Systems upgrading

    From Wesley@21:1/5 to All on Tue Aug 20 17:30:02 2024
    Hello

    We have 2000+ dedi servers planning to upgrade from debian 11 to Debian
    12.

    The main tech stacks on them are apache, mysql, redis, resin/tomcat,
    ceph and some hdfs nodes.

    We are looking for a consultancy for the implementation. If there is any
    who have the interest, please contact me.

    Thanks.


    --
    https://wespeng.pages.dev/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Smith@21:1/5 to Wesley on Tue Aug 20 17:50:02 2024
    Hi Wesley,

    On Tue, Aug 20, 2024 at 11:29:26PM +0800, Wesley wrote:
    We have 2000+ dedi servers planning to upgrade from debian 11 to Debian 12.

    I should think that the typical debian-user subscriber manages only
    a small number of systems with all of them for personal/hobby use,
    with the rest of us being extreme outliers.

    I think your use case is vastly outside of the experience of almost
    everyone here, so here may not be a great place to find a
    consultant.

    This page may help in your search:

    https://www.debian.org/consultants/

    I can recommend some in my country (UK) but maybe that is not of
    interest to you.

    I've never used them, but my company pays Freexian at the bronze
    tier to do Debian LTS packaging work and I understand they do wider
    Debian consultancy, so maybe that is a good option.

    https://www.freexian.com/services/debian-support/

    There's the debian-consultants mailing list but it gets almost no
    traffic. I don't think it will aid you in finding a consultancy any
    better than the previous links, but just in case:

    https://lists.debian.org/debian-consultants/

    Just as some free advice though…

    1. I find it hard to believe you have more than 2000 Debian installs
    without some sort of existing automation / configuration
    management

    2. Given (1), I would approach the task by learning your config
    management and modifying it to deploy a Debian 12 version of each
    kind of Debian 11 server you already have.

    3. I'd then do a rolling deploy that slowly takes Debian 11 servers
    out of service and re-provisions them as Debian 12. I would not
    try to upgrade anything in place. Although Debian supports that,
    at scale I find it harder to account for all variables than with
    a clean install, and if you already have automation to deploy and
    configure a host then an in-lace upgrade also takes longer in my
    experience.

    Something to chat to your consultancy about, anyway.

    It would be really interesting if you or your consultant would write
    up what you ended up doing.

    Thanks,
    Andy

    --
    https://bitfolk.com/ -- No-nonsense VPS hosting

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Wesley@21:1/5 to Andy Smith on Tue Aug 20 18:30:01 2024
    And yes we just experienced a p0 accident which affected 100 millions
    end users. (There is maybe already the news on internet.) So we are
    taking serious consideration on hiring a consultant for our system
    upgrading.

    Thank you.

    On 2024-08-20 23:49, Andy Smith wrote:
    Hi Wesley,

    On Tue, Aug 20, 2024 at 11:29:26PM +0800, Wesley wrote:
    We have 2000+ dedi servers planning to upgrade from debian 11 to
    Debian 12.

    I should think that the typical debian-user subscriber manages only
    a small number of systems with all of them for personal/hobby use,
    with the rest of us being extreme outliers.

    I think your use case is vastly outside of the experience of almost
    everyone here, so here may not be a great place to find a
    consultant.

    This page may help in your search:

    https://www.debian.org/consultants/

    I can recommend some in my country (UK) but maybe that is not of
    interest to you.

    I've never used them, but my company pays Freexian at the bronze
    tier to do Debian LTS packaging work and I understand they do wider
    Debian consultancy, so maybe that is a good option.

    https://www.freexian.com/services/debian-support/

    There's the debian-consultants mailing list but it gets almost no
    traffic. I don't think it will aid you in finding a consultancy any
    better than the previous links, but just in case:

    https://lists.debian.org/debian-consultants/

    Just as some free advice though…

    1. I find it hard to believe you have more than 2000 Debian installs
    without some sort of existing automation / configuration
    management

    2. Given (1), I would approach the task by learning your config
    management and modifying it to deploy a Debian 12 version of each
    kind of Debian 11 server you already have.

    3. I'd then do a rolling deploy that slowly takes Debian 11 servers
    out of service and re-provisions them as Debian 12. I would not
    try to upgrade anything in place. Although Debian supports that,
    at scale I find it harder to account for all variables than with
    a clean install, and if you already have automation to deploy and
    configure a host then an in-lace upgrade also takes longer in my
    experience.

    Something to chat to your consultancy about, anyway.

    It would be really interesting if you or your consultant would write
    up what you ended up doing.

    Thanks,
    Andy

    --
    https://wespeng.pages.dev/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dan Ritter@21:1/5 to Andy Smith on Wed Aug 21 00:20:01 2024
    Andy Smith wrote:
    Just as some free advice though…

    1. I find it hard to believe you have more than 2000 Debian installs
    without some sort of existing automation / configuration
    management

    2. Given (1), I would approach the task by learning your config
    management and modifying it to deploy a Debian 12 version of each
    kind of Debian 11 server you already have.

    3. I'd then do a rolling deploy that slowly takes Debian 11 servers
    out of service and re-provisions them as Debian 12. I would not
    try to upgrade anything in place. Although Debian supports that,
    at scale I find it harder to account for all variables than with
    a clean install, and if you already have automation to deploy and
    configure a host then an in-lace upgrade also takes longer in my
    experience.

    We do hundreds rather than thousands, but we do them:

    - with an existing configuration automation system (chef/cinc)

    - in-place upgrades

    - in tiers, where a given function (e.g. web servers) will have
    representative machines in each tier, starting with a very
    small proof-of-concept upgrade, followed by corrections; then
    a somewhat larger upgrade group, followed by all the rest of
    the machines.

    Automated monitoring, too.

    -dsr-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)