SOLVED Migrated Proxmox VM won't start Asterisk

[kameleon]

Working on a setup for a customer that has an IncrediblePBX 16-15 install on a Centos7 vm. Before you go crazy, yes I know it needs to be upgraded and I am going to help them with that. The problem at hand is I am migrating them from their old Hyper-V 2019 server to a Proxmox 7 one. All of their VMs have migrated just fine by exporting the vm, setting up the new vm on proxmox, and importing the disk from the Hyper-V export in to Proxmox. With the only exception being the pbx one. Literally nothing changed between the export and import but when i start the vm it complains that "in start.class.php line 193 unable to connect to asterisk. did it start". Of course it did not but I also cannot find any reason why it did not. I have searched for a cause and possible solution and the only thing close I have found was someone saying that the AMI credentials were wrong but they are correct. I just need to get this vm migrated asap and then we can schedule a time to do an upgrade to a newer pbx. Anyone have an idea of where to start to find the problem?

 

[dallas]

Check the names of the Ethernet adapters in Proxmox and verify the ip address is correct in the VM.

 

[kameleon]

Check the names of the Ethernet adapters in Proxmox and verify the ip address is correct in the VM.

It is eth0 in both and the IP is the same. Only the uuid changed but I even tried changing it to the new name and same problem. I am beginning to wonder if it is a cpu compatibility issue. The old hyper-v server was a core i7 where the new proxmox one is a xeon. I'm probably going to have to leave the pbx on hyper-v for now and see when they can afford to upgrade the pbx. Then I'll just swap it out during the upgrade.

 

[KNERD]

If you think it is CPU compatibility issue, you can just recompile Asterisk..

Also "asterisk -c" will show you the startup output.

 

[wardmundy]

It is eth0 in both and the IP is the same. Only the uuid changed but I even tried changing it to the new name and same problem. I am beginning to wonder if it is a cpu compatibility issue. The old hyper-v server was a core i7 where the new proxmox one is a xeon. I'm probably going to have to leave the pbx on hyper-v for now and see when they can afford to upgrade the pbx. Then I'll just swap it out during the upgrade.

Try cleaning out the files in /etc/udev/rules.d and also in /var/lib/dhcp or /var/lib/dhcpd. Then reboot.

 

[kameleon]

I cleaned out the rules.d but did not have any folder dhcp or dhcpd. Reboot and same thing.

Running "asterisk -c" gets me this:

Running as user 'asterisk' Running under group 'asterisk' PBX UUID: ce539827-77f7-4bf6-bda9-78b77e45c812 [2023-03-04 22:42:27] NOTICE[4008]: loader.c:2377 load_modules: 338 modules will be loaded. [2023-03-04 22:42:27] NOTICE[4008]: loader.c:2377 load_modules: 338 modules will be loaded. [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-TravMan4' dsn->[MySQL-TravMan4] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-TravMan4' dsn->[MySQL-TravMan4] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asteridex' dsn->[MySQL-asteridex] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asteridex' dsn->[MySQL-asteridex] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asterisk' dsn->[MySQL-asterisk] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asterisk' dsn->[MySQL-asterisk] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asteriskcdrdb' dsn->[MySQL-asteriskcdrdb] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-asteriskcdrdb' dsn->[MySQL-asteriskcdrdb] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-mysql' dsn->[MySQL-mysql] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-mysql' dsn->[MySQL-mysql] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-performance_schema' dsn->[MySQL-performance_schema] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-performance_schema' dsn->[MySQL-performance_schema] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-test' dsn->[MySQL-test] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-test' dsn->[MySQL-test] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-timeclock' dsn->[MySQL-timeclock] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'MySQL-timeclock' dsn->[MySQL-timeclock] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'asteriskcdrdb' dsn->[MySQL-asteriskcdrdb] [2023-03-04 22:42:27] NOTICE[4008]: res_odbc.c:706 load_odbc_config: Registered ODBC class 'asteriskcdrdb' dsn->[MySQL-asteriskcdrdb] [ Initializing Custom Configuration Options ] [ Initializing Custom Configuration Options ] [2023-03-04 22:42:27] NOTICE[4008]: cdr.c:4501 cdr_toggle_runtime_options: CDR simple logging enabled. [2023-03-04 22:42:27] NOTICE[4008]: cdr.c:4501 cdr_toggle_runtime_options: CDR simple logging enabled. ............................................ [2023-03-04 22:42:27] WARNING[4008]: presencestate.c:200 ast_presence_state_helper: No provider found for label: CustomPresence [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'from-internal-additional' tries to include nonexistent context 'app-recordings' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'from-internal-additional' tries to include nonexistent context 'app-recordings' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'paging-chanspy' tries to include nonexistent context 'paging-chanspy-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'paging-chanspy' tries to include nonexistent context 'paging-chanspy-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-builddshunt' tries to include nonexistent context 'dialparties-builddshunt-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-builddshunt' tries to include nonexistent context 'dialparties-builddshunt-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-checkcfextension' tries to include nonexistent context 'dialparties-checkcfextension-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-checkcfextension' tries to include nonexistent context 'dialparties-checkcfextension-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-checkcfextensions' tries to include nonexistent context 'dialparties-checkcfextensions-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'dialparties-checkcfextensions' tries to include nonexistent context 'dialparties-checkcfextensions-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'sub-send-obroute-email' tries to include nonexistent context 'sub-send-obroute-email-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'sub-send-obroute-email' tries to include nonexistent context 'sub-send-obroute-email-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'from-trunk-sip-IPComms' tries to include nonexistent context 'from-trunk-sip-IPComms-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'from-trunk-sip-IPComms' tries to include nonexistent context 'from-trunk-sip-IPComms-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'ext-did-0002' tries to include nonexistent context 'ext-did-0002-custom' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'ext-did' tries to include nonexistent context 'ext-did-0001' [2023-03-04 22:42:27] WARNING[4008]: pbx.c:8732 ast_context_verify_includes: Context 'park-hints' tries to include nonexistent context 'parkedcalls' [2023-03-04 22:42:27] ERROR[4008]: res_config_pgsql.c:1606 pgsql_reconnect: PostgreSQL RealTime: Failed to connect database asterisk on 127.0.0.1: [2023-03-04 22:42:27] WARNING[4008]: res_config_pgsql.c:1544 parse_config: PostgreSQL RealTime: Couldn't establish connection. Check debug. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1247 load_mysql_config: MySQL RealTime: No database user found, using 'asterisk' as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1253 load_mysql_config: MySQL RealTime: No database password found, using 'asterisk' as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1259 load_mysql_config: MySQL RealTime: No database host found, using localhost via socket. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1265 load_mysql_config: MySQL RealTime: No database name found, using 'asterisk' as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1271 load_mysql_config: MySQL RealTime: No database port found, using 3306 as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1283 load_mysql_config: MySQL RealTime: No database socket found, using '/var/lib/mysql/mysql.sock' as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1283 load_mysql_config: MySQL RealTime: No database socket found, using '/var/lib/mysql/mysql.sock' as default. [2023-03-04 22:42:27] WARNING[4008]: res_config_mysql.c:1288 load_mysql_config: MySQL RealTime: No database socket found (and unable to detect a suitable path). [2023-03-04 22:42:27] NOTICE[4008]: dnsmgr.c:493 do_reload: Managed DNS entries will be refreshed every 300 seconds. [2023-03-04 22:42:27] NOTICE[4008]: dnsmgr.c:493 do_reload: Managed DNS entries will be refreshed every 300 seconds. [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind http server to 127.0.0.1:8088: Address already in use [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind http server to 127.0.0.1:8088: Address already in use [2023-03-04 22:42:27] WARNING[4008]: http.c:2216 __ast_http_load: Failed to start HTTP server for address 127.0.0.1:0 [2023-03-04 22:42:27] WARNING[4008]: http.c:2216 __ast_http_load: Failed to start HTTP server for address 127.0.0.1:0 [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind https server to [::]:8089: Address already in use [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind https server to [::]:8089: Address already in use [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind AMI server to 0.0.0.0:5038: Address already in use [2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind AMI server to 0.0.0.0:5038: Address already in use Illegal instruction


I checked that mysql was running and it is. Odd.

 

[Samot]

FreePBX uses AMI to perform backend actions. When that error is encountered it generally means that Asterisk is either not running or that AMI user for FreePBX doesn't have permissions (i.e Asterisk started as root). The latter is often the issue more than not. In a build like this it can generally mean Asterisk is loading *before* FreePBX and FreePBX cannot start Asterisk or get anything to bind to what it needs because *Asterisk is already running*.

[2023-03-04 22:42:27] ERROR[4008]: tcptls.c:770 ast_tcptls_server_start: Unable to bind AMI server to 0.0.0.0:5038: Address already in use

There are quite a few various errors like this in your output. It means that 127.0.0.1:8088 and the AMI port is already bound to something.

Side Note: res_config_mysql and res_config_pqsql are both RealTime modules which means for some reason this is being built with RealTime. Something that neither FreePBX or IPBX use so installing it, while it doesn't nothing bad, just means more things are being loaded into memory that aren't being used.

 

[KNERD]

Illegal instruction

Try recompiling Asterisk

 

[kameleon]

Try recompiling Asterisk

This was the answer! Recompiled and now everything works as intended. Thanks for that.

 

[KNERD]

Good to hear