Containerized architecture implementation topics

Overview

Our clients typically implement virtual machines, within which they must create an environment to host and manage the containers. When implementing the Classic LumenVox product line, there were no containers and no need for special accommodation. However, with this new architecture you need container orchestration software. LumenVox leverages the power of Kubernetes as a primary supported containerized orchestration system.  

Benefits

The new containerized micro-service architecture provides the following benefits:

• Auto scaling

Automatic scaling of resources allows the automatic increase or decrease of the number of containers that are running to meet capacity needs when workload changes. The system can easily handle peak times, while allowing you to save resources when they are not needed.

• Self-healing 

Using Kubernetes, containers will automatically be replaced should one fail. This also assists with version rollbacks and makes upgrades easier.

• Automated Disaster Recovery

Fail-over / disaster recovery (DR) – is made possible using the Kubernetes architecture.

• Easily create a local or global installation

After the system is in place you can scale up, add nodes or pods, and make it globally available.

• Platform independence with flexible deployment

Our products are now built as cloud-native and can be deployed on any Kubernetes system on your compute platform, be it on-premises, or any major cloud provider environment. You have the flexibility to set up your Kubernetes cluster in a private or public cloud, or in multi-cloud or hybrid environments.

• Easy to integrate with our other products

LumenVox entire technology suite has been converted to this architecture, which makes deployment of additional products much faster. You can add CPA or TTS to ASR, for example, and once you have speech products in place, you can add Voice Biometrics with greater ease.

• Resources and tools to get up and running

LumenVox’ integration with Kubernetes is provisioned using Helm charts that streamline the installation. In addition, sample test scripts and built-in diagnostics mean you can verify that everything works, allowing you to hit the ground running.

• Lower cost of ownership

Clients can make use of their existing hardware, and the solution utilizes popular and well supported, open-source software components like MongoDB and PostgreSQL instead of the more expensive Microsoft SQL and Oracle database technologies, for maintaining the installation and its data.

Implementation Options for Container Orchestration

Small Node-Count Kubernetes

For small operations, typically on-premises, or for a testing proof-of-concept exercise, Kubernetes in a minimal configuration is a good option.  The Speech products, using any of the communication protocols, can support several hundreds of concurrent users in this configuration, depending on hardware. Our Sales Engineering can assist with hardware sizing guidance and provide a recommended installation process.

Full Kubernetes Cluster

Kubernetes is a portable open-source platform that manages containerized workloads and services and is often called a container orchestration system.  It is used to automate cloud software deployments and scaling of containers, pods, and clusters (a pod is one or more containers). Kubernetes typically also provides for security, load balancing, automatic failover recovery, and self-healing. 

Kubernetes can be deployed in multiple environment configurations, including a public cloud. LumenVox has put together an environment on GCP (Google Cloud Platform) as an example for a Kubernetes environment that customers can access for assessment and testing.

Authentication

The containers are designed to be running inside a Kubernetes cluster. System hosts may provide security for this environment, or administrators may wish to utilize their own identification and authorization mechanisms.

In any case, LumenVox recommends following industry-standard best practices as well as following Kubernetes security best practices. This includes securing cluster nodes and other services, using appropriate firewalls, and VPC configurations as needed.

Environment 

The solution requires an environment in which Kubernetes can be installed - this is typically a Linux-based environment, which offers optimal performance. Clients should consider separating test and production systems. It is recommended that the Redis, RabbitMQ, MongoDB and PostgreSQL components be provisioned outside of the Kubernetes cluster for performance and resiliency purposes. 

Hardware

Sizing for production will need to be determined based on your specific cluster requirements. Please contact LumenVox to assist with sizing design.

Load balancing

The Kubernetes technology has its own form of load balancer built in. In addition, LumenVox also utilizes an advanced “Service-Mesh” mechanism to optimize traffic and security within its containerized environment.

Other Implementation Considerations

Software and Activities in a Production Environment

Some or all the following activities will take place in a production environment and are customer or partner responsibility:

• Provision of required hardware & host containerization software
• Installation of LumenVox software, and integrating the customer system for audio input into LumenVox and data output back into the customer speech application
• Setup of Kubernetes, RabbitMQ, Redis, MongoDB, and PostgreSQL
• Monitoring of hardware, software, and services e.g., by using tools like Prometheus, Grafana, or others
• Analysis and consolidation of logs e.g., by using log analysis tools like Datadog, Splunk, and many others
• Stress testing the full solution in a production environment
• Monitoring network and component latency
• Database management including scheduling of database cleanups
• Backup management

LumenVox Native Formats for Audio 

Audio must be recorded in one of the following formats and converted & submitted as the following types of headerless byte-streams:

• Linear signed PCM - 16-bit 8kHz sample rate 
• alaw compressed 8-bit 8kHz sample rate
• ulaw compress 8-bit 8kHz sample rate

WAV formats are also supported.

Security

At a glance, LumenVox data is secure both at rest and when in transit. We encrypt data with keys only available to customers, and those keys are further encrypted. We operate inside secure environments that have protection from outside access. We design URL and port usage in a manner that allows segregation of duties. For our integration with external components, we enable uploading and working with security certificates. We consider security end-to-end.

In more detail, LumenVox provides the following security features:

• The software utilizes secure TLS communication between components. TLS (Transport Layer Security) is a cryptographic protocol and is the successor of SSL. 
• The audio and data are encrypted at rest within the Mongo and PostgreSQL databases. 
• Our PKI (Public Key Infrastructure) implementation manages digital certificates for the components and public key encryption. All security is managed by the customer, including customer-managed keys, which can be rotated whenever needed, and needn’t be external facing. 
• The administration portal has a dedicated port for the cluster admin, and a separate dedicated port for tenant deployment administrators. The URLs are constructed to easily enable segregation of duties with firewall access rules. Tenants are shielded separately from one another and from the cluster administration.
• The only external connection required is the ‘phone-home’ licensing service requirement to the LumenVox licensing server (collecting only product usage counts) and link to S3 to download ASR & TTS resources.

Migration from Existing Technology

As LumenVox has embraced a different technology there may be a learning curve for the client or partner that hasn’t been exposed to container technology. Containerization is being embraced as technology of the future by all cloud providers and large companies especially because of its benefits such as dramatic scalability and self-healing. The technology comes with a learning curve but LumenVox endeavors to provide documents and tools to assist clients with the implementation e.g. Helm charts. This package enables clients to configure and customize their systems in a consistent, repeatable manner. We also provide test scripts that enable clients to do a rapid test to ensure things are working. 

Visit the LumenVox knowledgebase, as well as our repositories on GitHub and our API documentation at https://developer.lumenvox.com.  Comprehensive and detailed information on LumenVox software implementation can be found in the LumenVox Containers  Implementation Guide.