VoIP Strategic Review
Back in the day, we did a VoIP Strategic Review for a Company that got acquired by Vonage.
1. Executive Summary
Joel Garcia of AllCode met with the Head of Network Design, of Company (Company) on July 30th and 31st to review Company’s VoIP architecture and 3rd party partnerships.
Company is new to the North American telecommunications market. Company’s current focus is on improving their port capacity and carrier diversity in the United States to reduce their overall dependency on Twilio and increase call quality.
AllCode has significant experience in telecommunications in the Americas. We have reasonably priced resources with expertise in telco matters ranging from negotiating legal and pricing matters with US carriers to performing interops and writing complex call routing scripts on Kamailio Sip proxies.
Given Company’s focus on North America, we recommend that Company engage AllCode to provide introductions to the following outbound and inbound DID providers in the United States.
North American Carrier Recommendations:
AllCode has had positive experiences in interacting with these carriers in the past three months. AllCode should work with Company to come up with a carrier requirements document that can be leveraged to determine the viability and correct interop test cases for each of these carriers. AllCode can then perform the interops for 2 of these carriers to enable Company to introduce these carriers to their Acision SIP Proxy Least Cost Routing (LCR) engine by the end of the year.
2. Current State of Affairs
Company provides VoIP solutions for EMEA, the US, and APAC. EMEA comprises 80% of the volume. The US comprises 10% of the volume. APAC comprises the remaining 10% of the volume.
In EMEA, Company runs two UK colocations, which are able to service EMEA to the tune of 2 million minutes per day. For EMEA, the following carriers are used: British Telecom, Vodafone, Twilio, and Level3. With the recent introduction of the Acision SIP and RTP Proxies, there is comfort with the current implementation.
In the US, Company run on Amazon Web Services (AWS). Company does 5 million minutes per month, but they are expecting to grow. For the US, the following carriers are used: Tata, Twilio, and Level3. Twilio does the majority of the volume. With Twilio, there have been voice quality issues. Twilio has tried to introduce Company to their dynamic routing, but the call quality has remained poor. To the best of our knowledge, Twilio does not use quality scores in their dynamic routing. Their dynamic routing is solely based upon cost. Level3 has been slow to fulfill Company’s requests. Company currently has 200 dedicated ports with Level3.
In APAC, Company does 5 million minutes per month. They are expecting growth in APAC. For APAC, Company utilizes Telegate, Tata, and Twilio. Telegate is an aggregator that runs calls through Telstra and Opus. Company does not have a relationship with Telstra and Opus.
Company does not have a solution in Latin America nor in the BRIC countries. For these countries, Company is looking to institute a process to test call termination and origination prior to opening these regions for Sales. Part of this process will involve identifying the most appropriate regional carrier partners.
2.1.2 Carrier On-boarding Process
Interop testing with potential carriers is improving, but is still not as efficient as it needs to be. In the past, the performers of the interops have tried to fix the issues when they’ve encountered them. This is a problematic approach because significant time may be spent fixing a small problem only to discover 3 test cases later that a blocker issue has been encountered that cannot be resolved. The process for interop tests now consists of running through the entire interop and then assessing whether this is viable.
2.1.3 Fraud Protection
Company does not currently have a formal mechanism for fraud protection. Fraud protection is dependent upon the carrier. Level3 and Twillio offer fraud protection. Fraud will increase as volume increases.
WebRTC represents an excellent opportunity for Company. In Company’s Contact World product, when an agent wants to make a call, they login to Salesforce to place a call through the web browser. The back end of Contact World calls the agent on their phone. This call can be placed via an inbound carrier, the PSTN, or Company’s 3rd party partner Thinking Phones. The agent answers the phone. Company then bridges the call with the intended recipient of the call.
Using WebRTC, Company can greatly reduce the cost of the inbound leg and simplify the process of enabling the agent to make a call. With WebRTC, the agent can simply place the call directly through the web browser eliminating some of the confusion with the agent placing the call in the web browser and then Company dialing the agent’s phone.
In addition, WebRTC also improves long distance call quality due to the ability to leverage superior codecs like Opus an iLBC. Less signaling/media hops will intervene in the call delivery, which translates into a more direct route.
Unfortunately, WebRTC has some serious issues. First, Company requires the client to use the web browser Chrome. Internet Explorer does not support WebRTC. There are plugins to add WebRTC support to IE, https://temasys.atlassian.net/wiki/display/TWPP/WebRTC+Plugins. Chrome and plugin solutions are problematic in a call center environment because agents are not able to install apps on their machines. Second, Company’s WebRTC is dependent upon a 3rd party solution. The options for handling WebRTC are Twilio and Acision. Twilio provides a working WebRTC solution that is priced well. However, since Twilio runs on AWS, in order for WebRTC to work for a specific client, all of the Amazon IP addresses need to be opened on the client’s firewall. Many clients will not want to open all of the Amazon IP addresses on their firewall. Acision hosts their WebRTC solution on AWS as well. Acision suffers from the same issues as Twilio with the Amazon IP addresses on the firewall.
Company has a nice Minimum Viable Product (MVP). This solution needs to be polished and presumably implemented in house as they scale up.
2.3 SIP Proxy
Company has licensed the Acision SIP Proxy, which is based upon Crocodile, to be the SIP Proxy in EMEA, the US, and APAC. The Acision SIP Proxy will enable these regions to have a functioning least cost routing solution basd upon the price of a variable length number of prefix in a DID, typically the first six digits of a phone number. The proxies are currently deployed as high availability pairs in active-passive mode. On paper, the proxies also support WebRTC to SIP.
The current SIP proxy only uses price to determine the appropriate carrier to route the call to. Company would like the main driver for carrier routing to be based upon quality instead of price. This poses an interesting problem because quality is subjective. Company has implemented a VoIP Monitor solution to detect the call quality of the carriers but has not figured out how to extract the data and feed it into their Sip Proxy to drive the carrier routing based upon quality.
3.1.1 Carrier Recommendations
AllCode would like to provide introductions to the following carriers for Company. AllCode would also like to perform the interops with these carriers for Company. For all of these carriers, we have done the interops before. We believe that we can deliver these carriers in a timely and cost effective manner.
North American Carrier Recommendations:
Latin American Carrier Recommendations:
We recommend these carriers because they all share the following common traits:
- We have engaged with these carriers and their call quality has been high.
- Their LCR rates are very competitive.
- Interop testing has been quite painless. Their support is responsive and professional.
- Load balancing and failover setup is flexible and powerful.
- We have not encountered any blockers on the legal side.
- They are willing to negotiate on price.
Company does not currently have a mechanism to perform pre-assessment of new customers for VoIP quality. AllCode can also assist with this initiative. We would put together a proposal for a solution based on visualware.com. Company would license software from visualware.com. AllCode would install it on AWS in the same VPC as the Company SIP/RTP network elements. Potential Company customers would run an online test using VisualWare to hit the server on AWS. The resulting metrics would determine the feasibility of the clients’ network connectivity to run VoIP at the desired scale.
3.1.2 Carrier On-boarding Process
For these region specific carriers, Company needs to put together a carrier requirements document. This document needs to outline the criteria a regional carrier needs to comply with to connect to the Company platform.
The carrier requirements document content would be driven by Company’s unique SIP equipment, signaling scenarios and business goals. Some requirements were mentioned in our meeting on July 30, 20XX. For example, the need to have support for a DID provisioning portal or a number provisioning API. Codec and DTMF requirements would need to be specified. Lower level SIP requirements could include support for empty SDP re-INVITE messaging when bridging calls. To determine lower level SIP requirements, AllCode would need to see Wireshark packet captures for common call scenarios. From these requirements and the Wireshark captures, we’d be able to draft a list of interop test cases.
Prior to doing the interop with a new carrier, our recommendation is to run the rate deck through a simulator to enable Company to see how much traffic the carrier would actually get if the carrier were placed into Company’s LCR. AllCode has experience in specifying and implementing this solution. Company would need to enhance their Midas solution to condense their monthly traffic into a minute total per NPA NXX. Next, Company would run the data through the LCR to see what carrier gets the traffic.
3.1.3 Fraud Protection
We would need to learn more about the existing Company fraud detection patterns. What are the fraud detection patterns that Company has observed so far? We would take these patterns and analyze them to develop a solution.
3.1.4 Profit Protection
This is a topic that would warrant further research to address the questions of what to do in order to preserve profit margins. Whatever technical solution is developed would need to be backed by suitable legal language in Company’s contract agreement, i.e. including a “fair use” clause when customers’ traffic patterns exceed Company’s acceptable operating costs.
In technical terms, Company would need to develop a real-time system to enable the flagging of accounts that spend more on telco charges than the total per set income. When these alerts are triggered, they should generate notifications. Conceivably, the accounts could be limited and the customers notified.
By leveraging an independent RespOrg and aggregator like NuWave or Hypercube, Company will be able to control inbound costs by routing to the cheapest IXC, e.g. Version, Level3, and AT&T. In the near term, when an IXC experiences an outage, Company would be able to change within 15 minutes the routing in the SMS/800, the US core toll free routing database, to have the calls to a given Toll Free Number (TFN) redirected to a working IXC. As it currently stands, when an IXC fails for an inbound TFN call, Company has no recourse.
AllCode has experience with both NuWave and Hypercube. AllCode would be able to introduce Company to NuWave and Hypercube to assess the feasibility of leveraging an independent RespOrg.
Company currently leverages Twilio for WebRTC. If Company has the desire to move away from Twilio, either to reduce their dependency or to obtain more control, there are a couple of possible solutions:
- Leverage the Acision hosted WebRTC solution.
- Channel the SIP signaling to Acision and the SRTP to an Asterisk server that AllCode would configure.
AllCode has resources that have implemented a WebRTC solution before with OpenSIPS and Asterisk. AllCode would be able to both write specifications and develop this solution if Company wanted to host the solution.
3.3 SIP Proxy
3.3.1 Carrier Capacity
It is our understanding that Company’s Acision SIP Proxy does not use carrier port capacity and calls per second to route traffic to other carriers. Company should introduce logic into the LCR so when a carrier’s max calls per second or calls in progress values are hit, the SIP Proxy LCR logic should overflow calls to the next cheapest carrier.
AllCode would be able to help with the specification for this feature.
Company is not able to see their telco spend in real-time. Unfortunately, the Acision LCR doesn’t contain the actual rate data. There are two alternatives:
- Put the telco rate into the LCR, which enables you to see cost in real-time.
- Dump the SIP Proxy CDRS in real-time to a message bus connected to a repository. Attach the repository to a real-time analytics package.
AllCode has experience in writing complex call routing and LCR scripts on Kamailio and OpenSIPS Sip proxies. We would be able to leverage this expertise in drafting requirements for Acision or creating a solution based upon the dumped SIP proxy CDRs.
3.3.3 SIP Monitor
As Company adds more carriers to the mix, there will be a desire see which carriers are getting the lion’s share of the traffic. AllCode can enable Company to know in real-time the distribution of their carrier traffic which will enable them to shape their traffic for cost or quality.
AllCode has written a SIP Monitoring solution in the past to view the calls per second and overall health of a carrier.
3.3.4 Call Routing Based Upon QoS as well as Cost
The Company LCR is based upon price not on call quality. Company wants to introduce VoIP monitoring carrier specific statistics into the LCR equation. The easiest approach would be to replace the $ LCR with a MOS based LCR score computed by VoIP Monitoring. The carriers would be ordered in descending order based upon MOS scores. The MOS score would be calculated at the NPA-NXX level using the following bit of SQL:
round (b_mos_min_mult10 /10,1) as called_mos,
count(*) / if(@cnt_all is null, @cnt_all := (select count(*) from cdr where b_mos_min_mult10 > 0), @cnt_all) * 100 as perc
AllCode would be able to help with the specification of the QoS LCR.
3.3.5 LNP Dips
At high call volume, Company can reduce costs on outbound calls significantly by leveraging a Local Number Portability (LNP) dip. In the United States, even if the carrier provides a rate deck, the rate may not be used to terminate the call as a different carrier still may own the number. In order to determine what the true rate is, the number must be dipped with a 3rd party service such as Novatel or Optimal Path. These services cost money to dip, $6k per month, but can significantly reduce costs. Typically, changes to the SIP Proxy will be done to get the Proxy to dip the number prior to hitting the LCR. The typical cost savings at high outbound volume is 10-15%.
AllCode would be able to help with the specification or implementation of the LNP Dip.
3.3.6 Interstate and Intrastate
To the best of our knowledge, the Acision LCR does not differentiate between interstate and intrastate rates. When the call is in the SIP Proxy, it is key to determine whether the call is interstate or intrastate to pick up the correct carrier.
AllCode would be able to help with the specification of this feature.
Traditional IT vs. AWS – How Small Businesses can Benefit
AWS solutions can accomplish a variety of problems and tasks including IT needs. Even smaller businesses that have a more limited scope that their businesses cover can look to find some way to upgrade their business operations through what Amazon has to offer. Though it may be intimidating and difficult to adapt to, there is more than enough reason to adopt AWS.
AWS Think Big with Small Business Program’s Competitive Edge
The Amazon Cloud can help alleviate most issues involving transitioning the cloud. Businesses of any scope can hope to build solutions that are scalable and adaptable to their industry of work. Smaller or minority-owned businesses may still struggle to stand out among bigger companies or make an AWS environment as efficient as possible with fewer funds to spend. To alleviate these burdens, Amazon has the Think Big With Small Business program available through their Public Sector partnership program.
AWS Think Big for Small Business, Data Analytics, and Business Intelligence
The AWS Think Big for Small Business Program is an outreach program designed to provide small and/or minority-owned public sector organizations support in the form of business intelligence, technical expertise, and marketing strategies. With cloud-based solutions and experience, various public institutions globally have seen continued success in government, educational, and nonprofit sectors. While the funding provided can help significantly to meet business objectives, the expertise on navigating the cloud and how to extend outwards towards customers is just as critical.
Joel Garcia has been building AllCode since 2015. He’s an innovative, hands-on executive with a proven record of designing, developing, and operating Software-as-a-Service (SaaS), mobile, and desktop solutions. Joel has expertise in HealthTech, VoIP, and cloud-based solutions. Joel has experience scaling multiple start-ups for successful exits to IMS Health and Golden Gate Capital, as well as working at mature, industry-leading software companies. He’s held executive engineering positions in San Francisco at TidalWave, LittleCast, Self Health Network, LiveVox acquired by Golden Gate Capital, and Med-Vantage acquired by IMS Health.