open https://gitlab.synchro.net/main/sbbs/-/issues/324

At least since 3.19b, a clean install (Debian Stretch in Docker) will crash at startup.Start a built image in bash:```> docker exec --rm -it bbsio/synchronet:3.19b bash```Run the `sbbs-init` to ensure the node directories etc are extracted (note, also downloads x00, but unrelated).Run `sbbs`, will start, and summarily crash.Will attempt to gather a GDB capture and attach additional details... logs below.[Dockerfile](https://github.com/bbs-io/synchronet-docker/blob/master/docker/Dockerfile)-----```~/sbbs took 4m15s ❯ docker run --rm -it bbsio/synchronet:3.19b bashUnable to find image 'bbsio/synchronet:3.19b' locally3.19b: Pulling from bbsio/synchronetDigest: sha256:891ef00100776e2efc32f0a7667be1928d569891cd93f99eb48c3849d7d6e1a7Status: Downloaded newer image for bbsio/synchronet:3.19broot@2216866b078f:/sbbs/ctrl# sbbs-initDownloading and extraction x00 1.53a--2022-01-21 22:51:59-- https://s1.bbs.land/bbs-io/downloads/fossil_drivers/x00153a.zipResolving s1.bbs.land (s1.bbs.land)... 205.185.216.10, 205.185.216.42Connecting to s1.bbs.land (s1.bbs.land)|205.185.216.10|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 108621 (106K) [application/x-zip-compressed]Saving to: 'x00153a.zip'x00153a.zip 100%[===========================================================================================>] 106.08K --.-KB/s in 0.05s 2022-01-21 22:51:59 (1.93 MB/s) - 'x00153a.zip' saved [108621/108621]root@2216866b078f:/sbbs/ctrl# sbbsSynchronet Console for Linux-x64 Version 3.19b Copyright 2022 Rob SwindellReading /sbbs/ctrl/sbbs.iniLoading configuration files from /sbbs/ctrl1/21 22:52:02 stat Loading configuration files from /sbbs/ctrl1/21 22:52:02 ftp Synchronet FTP Server Version 3.19b1/21 22:52:02 ftp Compiled HEAD/6b10fdc Jan 3 2022 15:31:17 with GCC 6.3.01/21 22:52:02 mail Synchronet Mail Server Version 3.19b1/21 22:52:02 term Synchronet Terminal Server Version 3.19b1/21 22:52:02 ftp Initializing on Fri Jan 21 22:52:02 2022 with options: 14WARNING: No user

account specified, running as root!1/21 22:52:02 mail Compiled HEAD/6b10fdc Jan 3 2022 15:31:22 with GCC 6.3.01/21 22:52:02 ftp Loading configuration files from /sbbs/ctrl 1/21 22:52:02 term Compiled HEAD/6b10fdc Jan 3 2022 15:30:51 with GCC 6.3.01/21 22:52:02 srvc Synchronet Services Version 3.19b 1/21 22:52:02 mail Initializing on Fri Jan 21 22:52:02 2022 with options: 600004041/21 22:52:02 web Synchronet Web Server Version 3.19b 1/21 22:52:02 web Compiled HEAD/6b10fdc Jan 3 2022 15:31:32 with GCC 6.3.01/21 22:52:02 mail Loading configuration files from /sbbs/ctrl 1/21 22:52:02 term Initializing on Fri Jan 21 22:52:02 2022 with options: 10221/21 22:52:02 srvc Compiled HEAD/6b10fdc Jan 3 2022 15:31:28 with GCC 6.3.01/21 22:52:02 term Loading configuration files from /sbbs/ctrl 1/21 22:52:02 web Initializing on Fri Jan 21 22:52:02 2022 with options: 8a01/21 22:52:02 srvc Initializing on Fri Jan 21 22:52:02 2022 with options: 8001/21 22:52:02 web Loading configuration files from /sbbs/ctrl 1/21 22:52:02 srvc Loading configuration files from /sbbs/ctrl 1/21 22:52:02 stat Listening on /sbbs/ctrl/status.sock 1/21 22:52:02 ftp FTP Server listening on socket 0.0.0.0 port 21 1/21 22:52:02 mail SMTP Transfer Agent listening on socket 0.0.0.0 port 25[Threads: 8 Sockets: 1 Clients: 0 Served: 0 Errors: 0] (?=Help): Illegal instruction (core dumped)root@2216866b078f:/sbbs/ctrl# ```
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2206

That capture says "core dumped". Do you have the core file?
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2207

@rswindell I can pull it out or recreate it easily enough if you like... where should it be located?
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2211

FWIW, I've seen this `Illegal instruction` before - and in my case it was CPU related.IE: If I build SBBS on my Intel(R) Core(TM) i7-10710U, but run it on my AMD G-T40E Processor, I get the Illegal instruction everytime. But if I build it on the AMD it works without problem.So not sure if this helps your case - be good to know why, but I'm betting its some thing to do during the compile with the different CPUs?
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Deon George wrote to GitLab note in main/sbbs <=-

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2211
FWIW, I've seen this `Illegal instruction` before - and in my case it
was CPU related.
So not sure if this helps your case - be good to know why, but I'm
betting its some thing to do during the compile with the different
CPUs?

I think when I looked into this, it was the cryptlib build that caused
it.

Changing the tools/ccopt.sh script to use -march=x86-64 instead of -march=native fixed it for me.

If I understand it correctly, native says use all of the features of the current architechture/cpu. Which might not be something you want in a
docker build. ;)

Take care,
Steve (bugz)

... Direct from the Ministry of Silly Walks
--- MultiMail/Linux v0.52
� Synchronet � Red Green BBS (bbs.red-green.com)
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2243

@leenooks that's entirely possible, the image(s) are being built on Github's via daily actions. Not sure if it's maybe using default optimizations that shouldn't be used for cross-platform. AFAIK, the build environmnet is Intel Xeon based and locally (and in Digital Ocean) I'm running on AMD.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2310

Pretty sure this is specific to your use of Docker. Re-open if you think overwise.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

close https://gitlab.synchro.net/main/sbbs/-/issues/324
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2317

Hey Rob, I think your assessment here is incorrect.So just now, I did this - outside of docker (ie: no docker involved).On my `Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz` I build SBBS, using the instructions [here](http://wiki.synchro.net/install:nix) (using the instructions under `Getting/Building`. My build command was `make install SYMLINK=1 NOCAP=1 DEBUG=1`.My build was done on a shared filesystem with the target system a `AMD G-T40E Processor`.After the build, on that system I ran `SBBSCTRL=`pwd`/ctrl valgrind exec/sbbs`, which resulted in this:```==1143855== Memcheck, a memory error detector==1143855== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.==1143855== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info==1143855== Command: exec/sbbs==1143855==Synchronet Console for Linux-x64 Version 3.19c Copyright 2022 Rob SwindellReading /gfs/home/devel/sbbs/ctrl/sbbs.iniLoading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:05 term Synchronet Terminal Server Version 3.19c Debug2/24 22:47:05 term Compiled master/7e5ab9a Feb 24 2022 22:35:39 with GCC 4.8.52/24 22:47:05 term Initializing on Thu Feb 24 22:47:05 2022 with options: 10222/24 22:47:05 term Loading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:04 stat Loading configuration files from /gfs/home/devel/sbbs/ctrl!Started as non-root user (id 500): May fail to bind TCP/UDP ports below 10242/24 22:47:05 srvc Synchronet Services Version 3.19c Debug2/24 22:47:05 srvc Compiled master/7e5ab9a Feb 24 2022 22:35:55 with GCC 4.8.52/24 22:47:06 ftp Synchronet FTP Server Version 3.19c Debug2/24 22:47:06 ftp Compiled master/7e5ab9a Feb 24 2022 22:35:53 with GCC 4.8.52/24 22:47:06 srvc Initializing on Thu Feb 24 22:47:06 2022 with options: 8002/24 22:47:06 srvc Loading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:06 web Synchronet Web Server Version 3.19c Debug2/24 22:47:06 web Compiled master/7e5ab9a Feb 24 2022 22:36:07 with GCC 4.8.52/24 22:47:06 web Initializing on Thu Feb 24 22:47:06 2

022 with options: 8a02/24 22:47:06 ftp Initializing on Thu Feb 24 22:47:06 2022 with options: 142/24 22:47:06 ftp Loading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:06 web Loading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:06 mail Synchronet Mail Server Version 3.19c Debug2/24 22:47:06 mail Compiled master/7e5ab9a Feb 24 2022 22:35:54 with GCC 4.8.52/24 22:47:06 mail Initializing on Thu Feb 24 22:47:06 2022 with options: 600004042/24 22:47:07 mail Loading configuration files from /gfs/home/devel/sbbs/ctrl2/24 22:47:07 stat Listening on /gfs/home/devel/sbbs/ctrl/status.sock2/24 22:47:07 term Configured time zone (UTC, 0x0000, offset: 0) does not match system-local time zone offset: 6602/24 22:47:08 term Verifying/creating data directories[Threads: 8 Sockets: 0 Clients: 0 Served: 0 Errors: 0] (?=Help): vex amd64->IR: unhandled instruction bytes: 0xC4 0xE2 0xC8 0xF2 0xF0 0x44 0x89 0xC0 0x48 0x85vex amd64->IR: REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONEvex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0==1143855== valgrind: Unrecognised instruction at address 0x4e8b6ec.==1143855== at 0x4E8B6EC: aclConsistent.isra.1.part.2 (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4E8B9D3: initAttributeACL (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4E4A388: krnlBeginInit (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4DB832C: initCryptlib (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4DB28D8: cryptInit (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4AE3777: internal_do_cryptInit (ssl.c:244)==1143855== by 0x64DD34E: __pthread_once_slow (pthread_once.c:116)==1143855== by 0x4AE37EA: do_cryptInit (ssl.c:258)==1143855== by 0x4AE3877: get_ssl_cert (ssl.c:290)==1143855== by 0x5E5A605: web_server (w

ebsrvr.c:7071)==1143855== by 0x64D4EA6: start_thread (pthread_create.c:477)==1143855== by 0x6918DEE: clone (clone.S:95)==1143855== Your program just tried to execute an instruction that Valgrind==1143855== did not recognise. There are two possible reasons for this.==1143855== 1. Your program has a bug and erroneously jumped to a non-code==1143855== location. If you are running Memcheck and you just saw a==1143855== warning about a bad jump, it's probably your program's fault.==1143855== 2. The instruction is legitimate but Valgrind doesn't handle it,==1143855== i.e. it's Valgrind's fault. If you think this is the case or==1143855== you are not sure, please let us know and we'll try to fix it.==1143855== Either way, Valgrind will now raise a SIGILL signal which will==1143855== probably kill your program.==1143855====1143855== Process terminating with default action of signal 4 (SIGILL)==1143855== Illegal opcode at address 0x4E8B6EC==1143855== at 0x4E8B6EC: aclConsistent.isra.1.part.2 (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4E8B9D3: initAttributeACL (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4E4A388: krnlBeginInit (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4DB832C: initCryptlib (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4DB28D8: cryptInit (in /gfs/home/devel/sbbs/repo/src/sbbs3/gcc.linux.x64.lib.debug/libsbbs.so)==1143855== by 0x4AE3777: internal_do_cryptInit (ssl.c:244)==1143855== by 0x64DD34E: __pthread_once_slow (pthread_once.c:116)==1143855== by 0x4AE37EA: do_cryptInit (ssl.c:258)==1143855== by 0x4AE3877: get_ssl_cert (ssl.c:290)==1143855== by 0x5E5A605: web_server (websrvr.c:7071)==1143855== by 0x64D4EA6: start_thread (pthread_create.c:477)==1143855== by 0x6918DEE: clone (clone.S:95)==1143855====1143855== HEAP SUMMARY:==1143855== in use at exit: 716,810

bytes in 5,456 blocks==1143855== total heap usage: 13,458 allocs, 8,002 frees, 2,079,527 bytes allocated==1143855====1143855== LEAK SUMMARY:==1143855== definitely lost: 24 bytes in 1 blocks==1143855== indirectly lost: 11 bytes in 2 blocks==1143855== possibly lost: 2,128 bytes in 7 blocks==1143855== still reachable: 714,647 bytes in 5,446 blocks==1143855== suppressed: 0 bytes in 0 blocks==1143855== Rerun with --leak-check=full to see details of leaked memory==1143855====1143855== For lists of detected and suppressed errors, rerun with: -s==1143855== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)Illegal instruction```It's a debug build, so hopefully that helps?
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

reopen https://gitlab.synchro.net/main/sbbs/-/issues/324

At least since 3.19b, a clean install (Debian Stretch in Docker) will crash at startup.Start a built image in bash:```> docker exec --rm -it bbsio/synchronet:3.19b bash```Run the `sbbs-init` to ensure the node directories etc are extracted (note, also downloads x00, but unrelated).Run `sbbs`, will start, and summarily crash.Will attempt to gather a GDB capture and attach additional details... logs below.[Dockerfile](https://github.com/bbs-io/synchronet-docker/blob/master/docker/Dockerfile)-----```~/sbbs took 4m15s ❯ docker run --rm -it bbsio/synchronet:3.19b bashUnable to find image 'bbsio/synchronet:3.19b' locally3.19b: Pulling from bbsio/synchronetDigest: sha256:891ef00100776e2efc32f0a7667be1928d569891cd93f99eb48c3849d7d6e1a7Status: Downloaded newer image for bbsio/synchronet:3.19broot@2216866b078f:/sbbs/ctrl# sbbs-initDownloading and extraction x00 1.53a--2022-01-21 22:51:59-- https://s1.bbs.land/bbs-io/downloads/fossil_drivers/x00153a.zipResolving s1.bbs.land (s1.bbs.land)... 205.185.216.10, 205.185.216.42Connecting to s1.bbs.land (s1.bbs.land)|205.185.216.10|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 108621 (106K) [application/x-zip-compressed]Saving to: 'x00153a.zip'x00153a.zip 100%[===========================================================================================>] 106.08K --.-KB/s in 0.05s 2022-01-21 22:51:59 (1.93 MB/s) - 'x00153a.zip' saved [108621/108621]root@2216866b078f:/sbbs/ctrl# sbbsSynchronet Console for Linux-x64 Version 3.19b Copyright 2022 Rob SwindellReading /sbbs/ctrl/sbbs.iniLoading configuration files from /sbbs/ctrl1/21 22:52:02 stat Loading configuration files from /sbbs/ctrl1/21 22:52:02 ftp Synchronet FTP Server Version 3.19b1/21 22:52:02 ftp Compiled HEAD/6b10fdc Jan 3 2022 15:31:17 with GCC 6.3.01/21 22:52:02 mail Synchronet Mail Server Version 3.19b1/21 22:52:02 term Synchronet Terminal Server Version 3.19b1/21 22:52:02 ftp Initializing on Fri Jan 21 22:52:02 2022 with options: 14WARNING: No user

account specified, running as root!1/21 22:52:02 mail Compiled HEAD/6b10fdc Jan 3 2022 15:31:22 with GCC 6.3.01/21 22:52:02 ftp Loading configuration files from /sbbs/ctrl 1/21 22:52:02 term Compiled HEAD/6b10fdc Jan 3 2022 15:30:51 with GCC 6.3.01/21 22:52:02 srvc Synchronet Services Version 3.19b 1/21 22:52:02 mail Initializing on Fri Jan 21 22:52:02 2022 with options: 600004041/21 22:52:02 web Synchronet Web Server Version 3.19b 1/21 22:52:02 web Compiled HEAD/6b10fdc Jan 3 2022 15:31:32 with GCC 6.3.01/21 22:52:02 mail Loading configuration files from /sbbs/ctrl 1/21 22:52:02 term Initializing on Fri Jan 21 22:52:02 2022 with options: 10221/21 22:52:02 srvc Compiled HEAD/6b10fdc Jan 3 2022 15:31:28 with GCC 6.3.01/21 22:52:02 term Loading configuration files from /sbbs/ctrl 1/21 22:52:02 web Initializing on Fri Jan 21 22:52:02 2022 with options: 8a01/21 22:52:02 srvc Initializing on Fri Jan 21 22:52:02 2022 with options: 8001/21 22:52:02 web Loading configuration files from /sbbs/ctrl 1/21 22:52:02 srvc Loading configuration files from /sbbs/ctrl 1/21 22:52:02 stat Listening on /sbbs/ctrl/status.sock 1/21 22:52:02 ftp FTP Server listening on socket 0.0.0.0 port 21 1/21 22:52:02 mail SMTP Transfer Agent listening on socket 0.0.0.0 port 25[Threads: 8 Sockets: 1 Clients: 0 Served: 0 Errors: 0] (?=Help): Illegal instruction (core dumped)root@2216866b078f:/sbbs/ctrl# ```
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2319

You're building sbbs for one platform/architecture and running on another. This (crash) is expected. Don't do that.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2328

It's expected for Synchronet? `arch` reports `x86_64` for both systems. I ask because I've installed both Linux (CentOS/Debian) and FreeBSD on these systems and each time I've used the same installation media and the systems have used the same upstream software repositories for applications and updates.I also run other software on both systems, and they came from the same upstream sources.Oh, and if I compile on the AMD, it runs fine on the Intel.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2329

Since, if I'm reading the backtrace correctly, its because of some crypto initialisation, and the Intel has the `aes` flag, whereas the `AMD` doesn't.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2330

I'm not seeing how this is a Synchronet issue. Build it on the system you intend to run it on: problem solved, no?
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2331

No, it doesn't solve it for me, but it is a workaround.The concept of building on an arbitrary system, and deploying on another is not new (given the same architecture). It's the basis of most operating systems and their applications that exist today. It's also exploited in the same way with docker based packaging of applications, but it isn't a "docker problem" IMHO when it fails. (You yourself offer that with your Windows installation?)It does put work back on the developer(s) to consider the different CPUs (when there are CPU dependant instructions) in their coding or building - but this is not a bad thing, it means you provide a more resilient product.It would be nice if this cause was identified so a programatic workaround could be implemented, but meh...
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2332

It works with Windows builds because I've configured the tool chain to always target i686-class (lowest common denominator) instruction set CPUs. e.g. the cryptlib library built by me for Windows does not take advantage of any new/fancy Intel or AMD instructions that may enhance crypto performance. You get that optimization in the *nix builds, but you also need to then build it on the same architecture that you're running on or otherwise take steps to explicitly specify the target architecture you *intend* to run it on.If anything, this is a cryptlib thing, not a Synchronet thing. Look into how cryptlib allows you to specify the target architecture. I'm sure if you can force it to target i686 or something similarly "old", it'll "just work" everywhere you want to run, though potentially not as fast as it possibly could.Again however, not a Synchronet issue. Maybe more of a documentation issue (wiki page update?).
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/324#note_2338

So it seems to be as a result of crypt lib's compile option `-march=native`. Removing that flag, I successfully build on the `Intel` and ran on the `AMD`.This should help with alleviate problems in a docker world, where the build environment is not necessarily the same as the target run environment, even though the architecture is the same.
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)