TIPS Suddenly could not log in to ipbx admin. Now navigating admin section is weird.

[JFrost]

Ok. So following the thread above that OPs problem and mine are not identical but suspiciously similar.

I do not have errors in freepbx_security.log but I do see some odd errors in

/var/log/asterisk/freepbx.log

For example, 'tail'ing the file, when I access the Admin -> Administrators section I see this in the log

[2021-10-12 18:18:47] [freepbx.INFO]: Depreciated Function module_getinfo detected in /var/www/html/admin/modules/core/page.ampusers.php on line 23 [] []

Then a few seconds later (not sure if related or 'random') I see this

[2021-10-12 18:19:01] [freepbx.INFO]: Deprecated way to add Console commands for module api, adding console commands this way can have negative performance impacts. Please use module.xml. See: https://wiki.freepbx.org/display/FOP/Adding+fwconsole+commands [] []
[2021-10-12 18:19:01] [freepbx.INFO]: Deprecated way to add Console commands for module backup, adding console commands this way can have negative performance impacts. Please use module.xml. See: https://wiki.freepbx.org/display/FOP/Adding+fwconsole+commands [] []
[2021-10-12 18:19:01] [freepbx.INFO]: Deprecated way to add Console commands for module voicemail, adding console commands this way can have negative performance impacts. Please use module.xml. See: https://wiki.freepbx.org/display/FOP/Adding+fwconsole+commands [] []

So then I access the User Manager page. On access, I see no errors.

However when I try to add a user in User Manager, on hitting submit I see the following errors

[2021-10-12 18:10:02] [freepbx.INFO]: Add user to letschat failed: The command "node /var/www/html/admin/modules/xmpp/node/adduserletschat.js eyJpZCI6IjEzIiwidXNlcm5hbWUiOiJuZXd1c2VyIiwiZGVzY3JpcHRpb24iOiJuZXcgdXNlciIsInBhc3N3b3JkIjoidGhpc2lzYXN0aWNrdXAiLCJlbmNyeXB0ZWQiOnRydWUsImV4dHJhRGF0YSI6eyJmbmFtZSI6IiIsImxuYW1lIjoiIiwidGl0bGUiOiIiLCJjb21wYW55IjoiIiwiZGVwYXJ0bWVudCI6IiIsImxhbmd1YWdlIjoiIiwidGltZXpvbmUiOiIiLCJ0aW1lZm9ybWF0IjoiIiwiZGF0ZWZvcm1hdCI6IiIsImRhdGV0aW1lZm9ybWF0IjoiIiwiZW1haWwiOiIiLCJjZWxsIjoiIiwid29yayI6IiIsImhvbWUiOiIiLCJmYXgiOiIiLCJkaXNwbGF5bmFtZSI6IiIsInByZXZFbWFpbCI6IiJ9LCJ1dWlkIjpudWxsLCJmcmVlcGJ4SWQiOiIxMyJ9" failed.  Exit Code: 1(General error)  Working directory: /var/www/html/admin  Output: ================   Error Output: ================ (node:21922) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.  /var/www/html/admin/modules/xmpp/node/node_modules/mongodb/lib/utils.js:123     process.nextTick(function() { throw err; });                                   ^ MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: letschat.users.$email_1  dup key: { : "" }     at Function.MongoError.create (/var/www/html/admin/modules/xmpp/node/node_modules/mongodb-core/lib/error.js:31:11)     at toError (/var/www/html/admin/modules/xmpp/node/node_modules/mongodb/lib/utils.js:139:22)     at /var/www/html/admin/modules/xmpp/node/node_modules/mongodb/lib/collection.js:748:67     at /var/www/html/admin/modules/xmpp/node/node_modules/mongodb-core/lib/connection/pool.js:469:18     at process._tickCallback (internal/process/next_tick.js:61:11)  [] []
[2021-10-12 18:10:15] [freepbx.INFO]: Deprecated way to add Console commands for module api, adding console commands this way can have negative performance impacts. Please use module.xml. See: https://wiki.freepbx.org/display/FOP/Adding+fwconsole+commands [] []

I have no idea how to interpret the above but it seems relevant. I wish Ward would chime in here somewhere. I'm sure he can easily answer some of the outstanding questions like:

Which PHP version should be for 2020?
What's the user/pass to access mysql?
Are these errors expected? If not, what do they indicate?

Hope we can get to the bottom of this soon.

 

[KNERD]

Logging in as root is probably disabled. You can "su asterisk" to log in to the SQL server

 

[KNERD]

PHP 5.3 should not be on it, and instead should be at least 5.6. Also "depreciated" just means the commands will be removed in the future., but yes it seems like a PHP issue.

 

[JFrost]

Logging in as root is probably disabled. You can "su asterisk" to log in to the SQL server

Tried to

 su -c mysql asterisk

and was asked for a password. I'm assuming its the Asterisk user pwd it wants but I don't have any records of setting one in my build log.
Tried the root pwd just in case but ofc that failed.

Thx for all the assist so far and sticking with me.

 

[KNERD]

password for the database should be in /etc/amportal.conf

 

[kenn10]

Did you try:

mysql -uasterisk -ppassw0rd

 

[JFrost]

password for the database should be in /etc/amportal.conf

Only password I found there is for AMPMGRPASS which did not work.
It looks like a hexadecimal number so it might be encoded somehow (not sure how to decode). The comment above said teh default is

amp111

and I tried that too but also failed.
From google I thought it would be actually AMPDBPASS but that is not in amportal.conf.

Anywhere else to look/get the db pass?

thx.

 

[JFrost]

Did you try:

mysql -uasterisk -ppassw0rd

Same result as with su - requires a password I do not know

(yes, I'm desperate enough that I even tried

passw0rd

which ofc, didn't work)

 

[JFrost]

According to Bill Simon here https://community.freepbx.org/t/i-need-information/73535

I should be able to access the db with the root user. Not sure why that fails.

EDIT - FOUND IT!!!

Not in amportal.conf but in

/etc /freepbx.conf

In there is AMPDBUSER and AMPDBPASS

 

[JFrost]

So an update -
After fighting for hours trying to get phpmyadmin installed on the box I gave up (would not install because php 5.6 is so far deprecated it cannot be made to use and I feared installing php 7.2 might overwrite or "steal" links the current UI uses.
I eventually settled on MySql Workbench (ugggh. Oracle) and after fighting it's bugs (crashing everytime I tried to connect) I got in.

I checked the 'admin', 'ampusers' and 'usermanusers' tables and all 3 appear to have all my users there.
Add'ly the 'usermanusers' table appears to have some of the test users I added during the troubleshooting the UI. So the DB seems to be fine and it's the Web UI that seems FUBAR.

At this point rather than blow away the whole box is it possible to install only the webui portions of IPBX?
If not possible to just "install" it in that sense, can I find somewhere the original webui files? I can try to do a diff against what's installed and/or just manually copy over the changes or whole thing.
It might solve the problem and it's worth a try I think before blowing the whole thing up.

Or, if anyone else has any ideas at this point, I'm wide open.

Thanks all.

 

[KNERD]

So an update -
After fighting for hours trying to get phpmyadmin installed on the box I gave up (would not install because php 5.6 is so far deprecated it cannot be made to use and I feared installing php 7.2 might overwrite or "steal" links the current UI uses.
I eventually settled on MySql Workbench (ugggh. Oracle) and after fighting it's bugs (crashing everytime I tried to connect) I got in.

I checked the 'admin', 'ampusers' and 'usermanusers' tables and all 3 appear to have all my users there.
Add'ly the 'usermanusers' table appears to have some of the test users I added during the troubleshooting the UI. So the DB seems to be fine and it's the Web UI that seems FUBAR.

At this point rather than blow away the whole box is it possible to install only the webui portions of IPBX?
If not possible to just "install" it in that sense, can I find somewhere the original webui files? I can try to do a diff against what's installed and/or just manually copy over the changes or whole thing.
It might solve the problem and it's worth a try I think before blowing the whole thing up.

Or, if anyone else has any ideas at this point, I'm wide open.

Thanks all.

Instead of phpmyadmin, you could install WebMin which has some similar functionality to it.

 

[ostridge]

Is that known (or how can I check/retrieve it), and/or how else can I log in to mysql db?

passw0rd with a zero

EDIT: missed your edit

 

[JFrost]

Instead of phpmyadmin, you could install WebMin which has some similar functionality to it.

Yeah that might've worked too. Either way, I got in.
The important thing was to be able to examine the tables, which I did, and all the users are in there.

Just the UI doesn't reflect that....

 

[JFrost]

I have tried to deconstruct the install scripts but I just cannot find where the actual (original) web files come from (those that would go to /var/www/html/admin)

I've also thought maybe it's worth a shot to do an in-place upgd from ipbx2020 to 2021 (if that's possible.) Gonna look for if there's instructions for that now..

 

[wardmundy]

Sounds to me like your MySQL asterisk database is hosed. You could try to fix it, but that requires access to MySQL. MORAL: That's why we make backups!

 

[JFrost]

Sounds to me like your MySQL asterisk database is hosed. You could try to fix it, but that requires access to MySQL. MORAL: That's why we make backups!

Why the database? What's the connection? (and if so, can I grab a copy of the default state of the DB somewhere?)

I did have Vultr taking automatic backups, I didn't think more than 2 weeks worth were necessary.

I also had some snapshots but just this month Vultr said they will start charging for them so we cleaned them up. The pbx had been working all that time (and still does) and I was in and using the interface inside the 2 weeks backup window with no sign of a problem. I only encountered the issue after rebooting my local machine and then trying to login to the console again. That was the first sign of an issue.


Edit to add:

but that requires access to MySQL

I do have access to the mysql database


EDIT 2 -
DB passed these integrity checks with no issue:

mysqlcheck -Av -u freepbxuser -p

mysqlcheck -Avg -u freepbxuser -p

(the -g in the second check is for version specific table issues)

There have been no "hard" downs on the server, no crashes, no power outages and no disk issues I'm aware of so there's little reason for the DB to have been corrupted and the integrity checks aren't flagging anything.

At this point I'm not sure I agree that it's the DB (though I could be wrong.) If there is some more thorough or specific check you can advise to run, I'm all ears.
At this point though my gut still feels more like a php interpreter or php files issue.
In all cases I still want to unravel this tangle, probably now more than when I started.

Thx.

 

[wardmundy]

The asterisk database stores most of the FreePBX data. If it gets corrupted, you'll see what you're seeing.

 

[JFrost]

The asterisk database stores most of the FreePBX data. If it gets corrupted, you'll see what you're seeing.

I understand that the DB contains the *data* (I'm fairly familiar with CMS/dynamic website dev) but does the DB in this case also contain the html and JS for the UI as well?
Because I'm not seeing corrupted *data*. I'm seeing a broken *website UI* that won't *display* that data.

The data on the website (where it's still shown) seems quite fully intact and visually examining the database shows all the data also quite reasonably intact (as best I can tell anyway). Now we also have run DB level integrity checks and they also indicate that the DB is intact.
For these reasons my gut says to look elsewhere.

Regardless, I'm willing to explore all possibilities. What you could help with here please is where can I get a "virgin" copy of the DB and where can I get same for the web UI files (php, html, js, etc..) that I can try to "diff"?
I have tried deconstructing the install scripts to find those but bash is not a language I read easily.

thx.

 

[wardmundy]

You can install a new version of Incredible PBX 2020 for CentOS 7 on Vultr in about 30 minutes. Just follow the tutorial here. It costs about a penny for 2 hours using the 1GB RAM size. Then you can extract anything desired from MySQL or elsewhere.

 

[JFrost]

You can install a new version of Incredible PBX 2020 for CentOS 7 on Vultr in about 30 minutes. Just follow the tutorial here. It costs about a penny for 2 hours using the 1GB RAM size. Then you can extract anything desired from MySQL or elsewhere.

So there's not a repo or a github I can grab them from?