NEW Incredible 16-15.2 for CentOS 7

[wardmundy]

By the way, a full FreePBX backup made on my 16-15.1 system did not properly restore on the 16-15.2 system. No voicemail, no CDR, no ring groups. Obviously, the FreePBX backup modules have a ways to go. I always backup using the incrediblebackup but the FreePBX backup was a good way to see if my primary system would restore to a fresh install. Surprisingly, the backup/restore works better on the backup from the legacy FreePBX 13 system.

We experienced the same thing.

 

[kenn10]

I just had Hosting73 move my cheap VPS to the Atlanta location. I'm going to try reloading 16-15.2 on this instance and see what happens.

 

[kenn10]

Installing 16-15.2 on a new VPS instance, the install is showing this:

Starting asterisk (via systemctl):
Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
[FAILED]
Restarting asterisk (via systemctl):  Warning: asterisk.service changed on disk. Run 'systemctl daemon-reload' to reload units.

The journalctl -xe shows this:

Sep 17 16:14:49 joecloud /usr/sbin/irqbalance[164]: WARNING, didn't collect load info for all cpus, balancing is broken
Sep 17 16:14:59 joecloud /usr/sbin/irqbalance[164]: WARNING, didn't collect load info for all cpus, balancing is broken
Sep 17 16:15:09 joecloud /usr/sbin/irqbalance[164]: WARNING, didn't collect load info for all cpus, balancing is broken
Sep 17 16:15:19 joecloud /usr/sbin/irqbalance[164]: WARNING, didn't collect load info for all cpus, balancing is broken
Sep 17 16:15:29 joecloud /usr/sbin/irqbalance[164]: WARNING, didn't collect load info for all cpus, balancing is broken

 

[wardmundy]

I've seen this once or twice. Only seems to appear with some hosting providers. Here's the ticket about it:

790899 – irqbalance: WARNING, didn't collect load info for all cpus, balancing is broken

bugzilla.redhat.com

 

[kenn10]

I've seen this once or twice. Only seems to appear with some hosting providers. Here's the ticket about it:

790899 – irqbalance: WARNING, didn't collect load info for all cpus, balancing is broken

bugzilla.redhat.com

Yeah. I found that. The solution doesn't work. That may be filling up a hidden logfile someplace and causing these issues. I saw another solution to disable IRBalance. I'll pursue further. This issue occurs of single CPU VPS's.

 

[wardmundy]

Yeah. I found that. The solution doesn't work. That may be filling up a hidden logfile someplace and causing these issues. I saw another solution to disable IRBalance. I'll pursue further. This issue occurs of single CPU VPS's.

Haven't seen this at all on Vultr boxes.

 

[kenn10]

@wardmundy on Hosting73 vm's, I can no longer install either 16-15.1 or 16-15.2. The install script hangs on any systemctl start xxxxx and all systemctl status xxxxx does not seem to return information the same way it did previously (never see the word "running" in the status.) I don't know if this is a new update that downloaded on the initial yum -y update command or not. The instance I am using was able to install and run 16-15.1 two days ago but can't now. I suspect a CentOS 7 "improvement."

Eventually, Asterisk and FreePBX install and will run but Fail2ban never starts and won't start manually. Phase one of the script runs fine but things go off the rails during the phase two install.

 

[wardmundy]

On the road today, but I'll have a look. Do the logs reveal anything unusual??

 

[dhoppy]

Has anyone tried the GUI backup to Dropbox? For me, it won't remember the storage location when I click save.

 

[kenn10]

Has anyone tried the GUI backup to Dropbox? For me, it won't remember the storage location when I click save.

That is going to be a FreePBX issue. Remember, that FreePBX 15 is still very much alpha/beta. Please report the issue on the FreePBX issue tracker website. I reported half a dozen issues with the backup module and they have been addressed and fixed within six weeks. If you don't report problems, they don't get fixed.

 

[kenn10]

On the road today, but I'll have a look. Do the logs reveal anything unusual??

Nothing really useful in the logs except that any systemctl service starts hang during the install and eventually time out. Apparently the services do actually start but the command doesn't know it. The same situation occurs on installation of 16-15.1 or 16-15.2. If I run the systemctl command from the command line, the same thing happens. I had this problem on the New York instance of Hosting73 and had them move it to Atlanta and have the same problem. I have a functioning 16-15.1 instance on another Hosting73 instance in Atlanta that is OK because it was installed a couple of weeks ago. Something has changed in the install since then whether a CentOS 7 "improvement" or an issue with the kernel or packages installed by the script.

There are various bug tracker tickets since 9/5/2019 about systemd hanging. See https://bugs.centos.org/view.php?id=16392

Looking at the fail2ban log file, the process is starting and stopping continually every one or two minutes.

Here are some log entries from phase-2 of the 16-15.1 install last night.

Installing file phoneprov/polycom_line.xml
Installing file phoneprov/snom-mac.xml
Starting asterisk (via systemctl):  Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
[FAILED]^M
Restarting asterisk (via systemctl):  Warning: asterisk.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
[FAILED]^M
.
.
.
make[2]: Leaving directory `/usr/src/mpg123-1.16.0'
make[1]: Leaving directory `/usr/src/mpg123-1.16.0'
Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
.
.
.
Updating / installing...
knock-server-0.5-7.el6.nux            warning: /etc/knockd.conf created as /etc/knockd.conf.rpmnew
########################################
Job for knockd.service failed because a timeout was exceeded. See "systemctl status knockd.service" and "journalctl -xe" for details.
.
.
.Installing file phoneprov/polycom.xml
Installing file phoneprov/polycom_line.xml
Installing file phoneprov/snom-mac.xml
Starting asterisk (via systemctl):  Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
[FAILED]^M
Restarting asterisk (via systemctl):  Warning: asterisk.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
[FAILED]^M
.
.
.
make[2]: Leaving directory `/usr/src/mpg123-1.16.0'
make[1]: Leaving directory `/usr/src/mpg123-1.16.0'
Job for asterisk.service failed because a timeout was exceeded. See "systemctl status asterisk.service" and "journalctl -xe" for details.
Ready to install FreePBX base platform...

 

[tbrummell]

Has anyone tried the GUI backup to Dropbox? For me, it won't remember the storage location when I click save.

I have something very similar. I cannot modify dial patterns in an Outbound Route or fix a typo in a hostname of the trunk. Make change, Apply, Commit changes. Stays the same. This is on 16-15.1

 

[wardmundy]

Sounds like I got the VirtualBox version of 16-15.2 built just in time.

 

[kenn10]

Sounds like I got the VirtualBox version of 16-15.2 built just in time.

You better believe I'm not touching my working production version of 16-15.1 on my other Hosting73 instance. I suspect if you did a fresh install from scratch on VULTR, you'll have the same problems after doing the yum update.

 

[hawk#1]

Sounds like I got the VirtualBox version of 16-15.2 built just in time.

Maybe I should make a back-up of the of the VirtualBox 16-15.2 system and restore it on a hosted system and see how that works.

 

[wardmundy]

Interesting that this only appears to be a problem with CentOS at the moment. Didn't see it with Scientific Linux 7.7 which we used for VirtualBox build.

I'm wondering if copying a working systemctl from another CentOS server will solve this?? If so, that's a pretty easy fix.

Also going to test removing all yum updates from the install process to see if that gets things going for the time being.

I'll give both a try.

 

[wardmundy]

We have successfully built an Incredible PBX 16-15.2 platform by NOT using yum update on the CentOS 7 platform. For the time being, this is probably the best solution, and we have modified the install script and tutorial accordingly.

 

[kenn10]

We have successfully built an Incredible PBX 16-15.2 platform by NOT using yum update on the CentOS 7 platform. For the time being, this is probably the best solution, and we have modified the install script and tutorial accordingly.

Didn't work for me. Still hangs on each systemctl start commannd until it times out.

 

[wardmundy]

@keenn10: Hosting73 may have already applied the updates to their CentOS 7 image.

On Vultr, systemctl --version shows: systemd 219 with:

+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN

 

[kenn10]

Same thing on Hosting73:

systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN

And my other Hosting73 instance that works has the same version.