TUTORIAL BulkVS SMS Reception/Transmission Gateway - php code

[Prits]

@Prits I apologize for my post earlier. I had started that before you posted that you are receiving email messages. If you are receiving email messages, my guess is that there is something potentially wrong with the url in bulkvs. Here is the url I have posted with them:

https://domain/sms/sms-rx.php

Good Luck! I hope you get it working soon.

no probs at all @smurfturf ! I am so grateful that you are trying to help etc

I too have posted similar URL at BulkVS, just that it contains path to my real domain name lol

NOTE: I have even added these IPs to the iptables as stated at BulkVS:

Inbound messages will come from one of the following IPs: 199.255.157.195 or 69.12.88.195 changed to

52.206.134.245 and 192.9.236.42 in Dec. 2021

 

[Halea]

Hello @Halea,

So I do use sendmail and my pbx does send emails out etc.

I just want to clarify, that I DO RECIEVE emails if I manually execute the script by calling the php file in a web browser ha

ie:

http://mypbx.com/sms-rx.php?from=0987456321&to=1234567890&message=Hello

I get the email, I sent it to you in a private conversation here with all the headers etc

Also, I do have the SMS enabled and my URL set at BulkVS

NOTE: I have even added these IPs to the iptables as stated at BulkVS:

Inbound messages will come from one of the following IPs: 199.255.157.195 or 69.12.88.195.

hmmmmm

If your PBX is behind a NAT type router, have you setup TCP port forwarding from your public IP to your PBX's private IP (for TCP#80)? Have you also allowed your local iptables firewall on the PBX to allow http connections to port 80 (since it's the one that you are using in your BulkVS config)?

 

[Prits]

If your PBX is behind a NAT type router, have you setup TCP port forwarding from your public IP to your PBX's private IP (for TCP#80)? Have you also allowed your local iptables firewall on the PBX to allow http connections to port 80 (since it's the one that you are using in your BulkVS config)?

@Halea

My IncrediblePBX is directly exposed to the PUBLIC...It is at CrownCloud VPS

I used ./add-ip to add these two BulkVS SMS gateway IP addresses with port 80 access to the IPTABLES, that BulkVS shows on their portal:

199.255.157.195 and 69.12.88.195

Update:

I moved the following scripts off the Incrediblepbx VPS at CrownClound:

sms-rx.php
sms-tx.php

to my web hosting platform and sms-rx.php works!

So, what gives on the Incrediblepbx server? lol

I have not done anything with sms-tx.php yet!

 

[kenn10]

If I am not mistaken, IncrediblePBX PUBLIC does not allow connection to port 80 unless you make modifications. I'm not sure that /root/add-ip will override that. The wiki covers this at https://wiki.incrediblepbx.com/article11-Incredible-PBX-2021-PUBLIC-Server.

 

[Halea]

If I am not mistaken, IncrediblePBX PUBLIC does not allow connection to port 80 unless you make modifications. I'm not sure that /root/add-ip will override that. The wiki covers this at https://wiki.incrediblepbx.com/article11-Incredible-PBX-2021-PUBLIC-Server.

It's probably the reason BulkVs is not able to deliver the sms messages to @Prits PBX.
So this is not an iptables issue but a restriction with the web server settings, which needs to be modified. Then iptables can be adjusted so that only the designated BulkVS IP address(es) can access the PBX web server on TCP port 80.
Good catch!

 

[jerrm]

t's probably the reason BulkVs is not able to deliver the sms messages to @Prits PBX.

If he's implemented the "public.conf" changes it could also explain the LetsEncrypt renewal issues he's had here.

If so, I would assume it was implemented after getting the initial cert.

 

[kenn10]

I wish people would actually specify exactly what version system they are running when asking for help. It would save a lot of tail-chasing when one thinks the OP's system is one thing and someone else thinks it is something else. @Prits did not mention he was running the PUBLIC version until message #23.

I guess we need to be interrogators before attempting to help.

 

[tbrummell]

I guess we need to be interrogators before attempting to help.

Never worked a technical customer (end user) support line before huh? The reported problem is *never* the actual problem. LoL
I worked at Mitel for 10 years doing pbx support for large corps. By the time a trouble ticket made it to us it was 3rd or 4th hand information, it just wasn't worth relying on what was in our ticket. LoL

 

[jerrm]

@Prits did not mention he was running the PUBLIC version until message #23.

Even then, there are different levels of "public." I had a good handle on what the core go-public script does to iptables/asterisk, but hadn't noticed the httpd suggestions in the newer articles.

If @Prits is using the httpd changes in the article, the bulkvs script fix could be as simple as dropping the scripts in the ucp folder. LetsEncrypt could be solved with symlinks in ucp or alias directives in the public.conf file.

 

[jerrm]

the bulkvs script fix could be as simple as dropping the scripts in the ucp folder. LetsEncrypt could be solved with symlinks in ucp or alias directives in the public.conf file.

Correcting myself here. I shouldn't think out loud..

Dropping extra files and symlinks in the ucp folder should work, but would cause (harmless) signature check messages (if enabled) and would be blown away the next time the ucp module updated.

Alias directives in public.conf are probably the best option.

 

[Prits]

I wish people would actually specify exactly what version system they are running when asking for help. It would save a lot of tail-chasing when one thinks the OP's system is one thing and someone else thinks it is something else. @Prits did not mention he was running the PUBLIC version until message #23.

I guess we need to be interrogators before attempting to help.

So sorry for any confusion @kenn10, in message #22, @Halea asked if I was behind a NAT, so I answered no, it is open to the PUBLIC at CrownCloud VPS. However, I did not mean to imply that I am running Incrediblepbx PUBLIC, I have not upgraded it to the 2021 - PUBLIC version as stated in this article:

Incredible PBX Wiki | Incredible PBX 2021 PUBLIC Server

Here is my system (so sorry that I did not post earlier, I am generally good about doing this):

Incredible PBX/FAX 2020.1 for CentOS 7

Asterisk: UP Apache: UP MySQL: UP
SendMail: UP IPtables: UP SSH: UP
LAN port: UP Fail2Ban: UP Webmin: UP
UCP Dmon: UP PortKnock: UP NR VPN: UP
FaxGetty: DN IAX Modem: DN HylaFax: DN

RAM:78MB CentOS Rel. 7.8.2003 Disk:8.4GB

Asterisk 16.12.0 Incredible GUI 15.0.12.40

Private IP: Hidden

Public Info: Hidden

System Time: Mon Jan 18 18:42:40 MST 2021

 

[Prits]

Even then, there are different levels of "public." I had a good handle on what the core go-public script does to iptables/asterisk, but hadn't noticed the httpd suggestions in the newer articles.

If @Prits is using the httpd changes in the article, the bulkvs script fix could be as simple as dropping the scripts in the ucp folder. LetsEncrypt could be solved with symlinks in ucp or alias directives in the public.conf file.

Sorry, @jerrm , this is my system at a CrownCloud VPS:

Incredible PBX/FAX 2020.1 for CentOS 7

Asterisk: UP Apache: UP MySQL: UP
SendMail: UP IPtables: UP SSH: UP
LAN port: UP Fail2Ban: UP Webmin: UP
UCP Dmon: UP PortKnock: UP NR VPN: UP
FaxGetty: DN IAX Modem: DN HylaFax: DN

RAM:78MB CentOS Rel. 7.8.2003 Disk:8.4GB

Asterisk 16.12.0 Incredible GUI 15.0.12.40

Private IP: Hidden

Public Info: Hidden

System Time: Mon Jan 18 18:42:40 MST 2021

 

[jerrm]

@Prits - at this point I would assume the issue is the http->https redirect you have setup in Apache. What EXACTLY is the url you have set up at bulkvs. If it is http://.... change to https://.... and open port 443 in iptables for the bulkvs api IPs.

Alternatively exclude the uri from the rewrite rule following the format I gave in the LetsEncrypt thread.

For anyone else following this thread, he didn't mention he has Apache set to redirect http to https.

 

[Prits]

@Prits - at this point I would assume the issue is the http->https redirect you have setup in Apache. What EXACTLY is the url you have set up at bulkvs. If it is http://.... change to https://.... and open port 443 in iptables for the bulkvs api IPs.

Alternatively exclude the uri from the rewrite rule following the format I gave in the LetsEncrypt thread.

For anyone else following this thread, he didn't mention he has Apache set to redirect http to https.

@jerrm, I had tried with both the following URLs at BulkVS:

http://mypbx.com/sms-rx.php

https://mypbx.com/sms-rx.php

Neither would work.

Having brought light to the fact that I had Apache set to re-direct all http to https, all I did was open up port 443 in the iptables for the BulkVS SMS gateway ip's as YOU had suggested and voila! IT WORKS!

Note, I did NOT have to exclude the URI from the rewrite rule either. So it remains intact.

(when I used ./add-ip to add the 2 BulkVS SMS gateway IPs, I failed to understand that the option I used had only opened up ports 80, 9080)

Thank you for your patience and HELP! ESPCIALLY with the update rewrite rule in the LetsEncrypt thread!

 

[Prits]

I posted the latest version of the scripts. Please take it down and unpack/decompress it. You'll find a text file that you should open a text editor (not a word processor though). Read through and you'll see that it is made of 2 parts. The first is for the reception script. The second for the transmission script.
You need to extract the first php block into a new text file that you call for instance sms-rx.php. The second php block goes into a second text file named sms-tx.php. A php block is what's between <?php and ?> (many lines later). It's really straight forward. Just copy and paste into a blank file.
Once you have your 2 files, edit them so that they contain your own personal data (like email addresses, BulkVS code, etc.) Each script has a section clearly marked as "personal/private data" where you need to enter your own.
Then you need to put the sms-rx.php file into the root directory of the PBX's web server. It will be /var/www/html, but might be different depending on your PBX software version or config. Finally to get it triggered, you need to go to BulkVS' settings panel and put-in your URL, which will look like: http://YourPBX.com/sms-rx.php
Note that you need to use your actual server's FQDN instead of YourPBX.com.
Try to get it going this far and let me know. With this much you will be able to receive sms messages sent to your BulkVS DID on your incoming email.
Then I'll guide you through the sms transmission script.

Hello @Halea ,

so I did what you told me... my files are called sms-rx.php and sms-tx.php

they are uploaded,

I added the URL to my BulkVS portal,

and I now [FINALLY] do receive an email messages in my inbox when the text message is sent!

So, I have made it this far, PART 1 is complete!

Now reporting back to you for PART 2 - the sms transmission

 

[Halea]

Good job @Prits ! I'm glad you got the sms to email part to work.

@kenn10 's and @jerrm 's help is invaluable as there are so many ends to master in order to get all the pieces to work together. IncrediblePBX is a complex mix of software to handle if it needs to be setup outside of its intended configuration.
When you need to do troubleshooting, it calls on every bit of knowledge and experience that you have accumulated over the years, and even that is often not enough, hence the need to follow closely forums such as this.

Regarding the firewalling matter that turned out to be the culprit (at least partially) in your case, I traditionally took a different approach on my home grown systems; I always dismantle all firewalling from the application servers like the PBX, web server, etc. Instead I use the firewall on my main router which for the last 10+ years has always been pfsense.

Now for the second part of the setup, install the sms-tx.php script where you've installed sms-rx.php.
(Make sure that the required php modules are installed on your PBX. The info is in the comments of the script file)

Then adjust its settings in the private/personal data section so that the mailbox which will be monitored is properly identified. There are 3 parameters for that; the server URL, the user id and the password. There is a gmail example which shows you what it's supposed to look like. The $hostname should include the full path to the IMAP incoming email box. It's not just the domain name. Again, look at the example.

The $AllowKey variable is a simple authorization code that you can set to anything you want. The example is "SendIt". What it means is that if you're not replying to an incoming message, thus making a fresh new outgoing text message, you need to have the variable KEY in addition to SMS_FR and SMS_TO entered on the subject line in the proper format (no space but the & sign as a separator, you should have KEY=SendIt if you keep the default authorization code) just like what the subject line structure looks like when you receive a message or reply to it.

Then you need to put in your BulkVS parameters API key and API secret. The URL is already in there and it's the same for all, so there is no need to change it.

Once the transmission script is properly configured, run the following command on the console of your PBX:
watch -n 10 "curl https://MyPBX.MyDomainName.com/sms-tx.php"
(Adjust the domain name and the URL to your PBX's path)
This will launch the script every 10 seconds, and if you replied to any incoming emailed sms message, or made an outbound one, it will process it and send it upstream through BulkVS' gateway.

If everything works the way it should, you can send to your BulkVS DID a text message from your cell phone and it will appear in your inbox almost instantly.
Then, when you reply to that email message, you'll see the reply text on your cell within 10 to 12 seconds.

To make the return faster you can reduce the script execution interval from 10s to 5s, but at some point the mailbox access timing will become the culprit. So, you can reasonably expect a 5 to 10 second responsiveness.
Now if the monitored mailbox is on your hardware within your control there are ways you can improve the responsiveness of everything and reduce the whole cycle to 1-2 seconds. That's the best I've achieved so far.

Ok, give it a try and see how it works for you. Keep us informed of your progress.
Once everything above worked for you, we'll trigger the periodic execution of the transmission script on your machine with a local cron job.

PS: Also, make sure that the reply-to address in the sms-rx.php script file corresponds to the email box that the transmission script sms-tx.php will be monitoring. Otherwise the reply message won't be properly routed.

 

[jaminmc]

This is Awesome! It would be better if you put it on Github. Then it is easy to change the code, and keep it upto date, and for us to see what is new in the code easily.

I did find a similar project on github. https://github.com/netsparkle/bulkvs-simple-sms

I like the watch -n 10 "curl https://MyPBX.MyDomainName.com/sms-tx.php" idea. But you would need a Screen session running to keep it going....

a cron entry like: * * * * * bash -c 'for i in {1..12}; do curl https://MyPBX.MyDomainName.com/sms-tx.php & sleep 5; done'
will spawn it every 5 seconds.

EDIT: encapsulated cron in bash -c '' & corrected my maths

 

[Halea]

...
I like the watch -n 10 "curl https://MyPBX.MyDomainName.com/sms-tx.php" idea. But you would need a Screen session running to keep it going....

a cron entry like: * * * * * for i in {1..30}; do curl https://MyPBX.MyDomainName.com/sms-tx.php & sleep 5; done
will spawn it every 5 seconds.

Yes, cron is certainly the best way once everything is properly configured and tested. The "watch -n 10 ..." method allows you to see the script's execution step by step especially if you turn on all the screen echos. I mostly use it for debugging. As you said, you need a terminal or console session to use it, but you can also put it in a bash file and push it to the background with a &. When I was developing this script I actually used my backup PBX server which has XFCE on it, so I had access to plenty of terminal sessions to play around.
I'm planning on making some improvements and then put it on github under GPL. Unfortunately I'm a bit busy right now with server deployments in colocation.

 

[Prits]

Good job @Prits ! I'm glad you got the sms to email part to work.

@kenn10 's and @jerrm 's help is invaluable as there are so many ends to master in order to get all the pieces to work together. IncrediblePBX is a complex mix of software to handle if it needs to be setup outside of its intended configuration.
When you need to do troubleshooting, it calls on every bit of knowledge and experience that you have accumulated over the years, and even that is often not enough, hence the need to follow closely forums such as this.

Regarding the firewalling matter that turned out to be the culprit (at least partially) in your case, I traditionally took a different approach on my home grown systems; I always dismantle all firewalling from the application servers like the PBX, web server, etc. Instead I use the firewall on my main router which for the last 10+ years has always been pfsense.

Now for the second part of the setup, install the sms-tx.php script where you've installed sms-rx.php.
(Make sure that the required php modules are installed on your PBX. The info is in the comments of the script file)

Then adjust its settings in the private/personal data section so that the mailbox which will be monitored is properly identified. There are 3 parameters for that; the server URL, the user id and the password. There is a gmail example which shows you what it's supposed to look like. The $hostname should include the full path to the IMAP incoming email box. It's not just the domain name. Again, look at the example.

The $AllowKey variable is a simple authorization code that you can set to anything you want. The example is "SendIt". What it means is that if you're not replying to an incoming message, thus making a fresh new outgoing text message, you need to have the variable KEY in addition to SMS_FR and SMS_TO entered on the subject line in the proper format (no space but the & sign as a separator, you should have KEY=SendIt if you keep the default authorization code) just like what the subject line structure looks like when you receive a message or reply to it.

Then you need to put in your BulkVS parameters API key and API secret. The URL is already in there and it's the same for all, so there is no need to change it.

Once the transmission script is properly configured, run the following command on the console of your PBX:
watch -n 10 "curl https://MyPBX.MyDomainName.com/sms-tx.php"
(Adjust the domain name and the URL to your PBX's path)
This will launch the script every 10 seconds, and if you replied to any incoming emailed sms message, or made an outbound one, it will process it and send it upstream through BulkVS' gateway.

If everything works the way it should, you can send to your BulkVS DID a text message from your cell phone and it will appear in your inbox almost instantly.
Then, when you reply to that email message, you'll see the reply text on your cell within 10 to 12 seconds.

To make the return faster you can reduce the script execution interval from 10s to 5s, but at some point the mailbox access timing will become the culprit. So, you can reasonably expect a 5 to 10 second responsiveness.
Now if the monitored mailbox is on your hardware within your control there are ways you can improve the responsiveness of everything and reduce the whole cycle to 1-2 seconds. That's the best I've achieved so far.

Ok, give it a try and see how it works for you. Keep us informed of your progress.
Once everything above worked for you, we'll trigger the periodic execution of the transmission script on your machine with a local cron job.

PS: Also, make sure that the reply-to address in the sms-rx.php script file corresponds to the email box that the transmission script sms-tx.php will be monitoring. Otherwise the reply message won't be properly routed.

Hello @Halea ,

So I have updated the sms-tx.php script as instructed.

I ran the following command on the console of the PBX (without the watch -n 10 just to see if it would work the first time) and here is the ouput:

root@pbx1:~ $ curl https://mydomain.tldt/sms-tx.php
curl: (60) Peer's Certificate issuer is not recognized.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
WARNING: Always run Incredible PBX behind a secure firewall.
root@pbx1:~ $

So then I ran this command and it's output:

root@pbx1:~ $ curl -k https://mydomain.tld/sms-tx.php
WARNING: Always run Incredible PBX behind a secure firewall.
root@pbx1:~ $

and nothing else happens, ie. sms text does not arrive and the reply/outbound email is still sitting in it's email box.

my system is still:

Incredible PBX/FAX 2020.1 for CentOS 7

Asterisk: UP Apache: UP MySQL: UP
SendMail: UP IPtables: UP SSH: UP
LAN port: UP Fail2Ban: UP Webmin: UP
UCP Dmon: UP PortKnock: UP NR VPN: UP
FaxGetty: DN IAX Modem: DN HylaFax: DN

RAM:94MB CentOS Rel. 7.8.2003 Disk:8.2GB

Asterisk 16.12.0 Incredible GUI 15.0.12.40

Private IP: <hidden>

Public Info: <hidden>

System Time: Sun Jan 24 15:45:12 MST 2021

 

[Halea]

I think I saw that before, the curl library needs to be configured with the proper CAs. I need to dig into my system and find out.
Meanwhile, if you are running this locally, or even if it's over the WAN, for now proceed with http instead of https and see if you can get everything else to work.
Ultimately you should run it with SSL but since you're still at the config phase bypass that issue for now and you'll fix it later.