/vnd/media/agency_attachments/bGjnvN2ncYDdhj74yP9p.png)
Over the last decade, the very nature of lawful monitoring has changed in India-moving from a structured content (signaling, phone records) to an unstructured content (web pages, emails, IM conversations). This leads to a radically different paradigm involving massive data storage and analysis that necessitates centralized deployment in response to this fundamental change.
The Challenges
Traditional database-driven BI solutions do not reflect the quantum leap in data analysis that has been triggered in the past few years by the monumental growth of Web 2.0 and cloud computing, manifested in the scientific and technical contributions of Google, Facebook, and Yahoo! to tackle big data. These companies have fundamentally changed the way quantitative analysis can be performed on gigantic data sets.
Communications is the social fabric that binds people together. Using technology, talking, meeting, chatting, and email have become the basic elements of a social network. Currently, deployed communication monitoring systems capture bits and bytes orphaned from the relationships that should inherently also be described to put together conversation threads with mobile phone locations that provide for context.
Applying Social Analytics
What is needed today is social analytics that applies social science to massive data sets using data-intensive supercomputing methods for statistical analysis using massively scalable computer-storage clusters, where the core on an open platform is flooded with petabytes of data on a daily basis. This will also lead to the use of predictive analytics.
Use of such techniques would lead to, for example, the following applications:
- Through a series of iterations across the entire data set of communications, captured data records could help to define a social graph-a way to describe the connections that bind people together. Unlike link analysis, this leads to defining and structuring relationships automatically over time that can help to detect popularity, alpha nodes in a group, anti-social behavior in clusters, degrees of separation between 2 targets, identify their associates, and more.
- Using sophisticated identity resolution process, it is also possible to correlate phones with similar calling patterns and associate them to a single identity.
- Captured calls can help to generate voice signatures, which can then use pattern-matching algorithms to detect callers using multiple SIM cards or across different media (VoIP) and associate them to the same node in the social network.
Quantum Leap Required
To lawfully monitor such communications, the surveillance industry today needs a quantum leap in capabilities to address the following 7 emerging trends:
- Massive growth in voice/mobile networks and explosion of the internet
- Daunting challenge to collect, store, and analyze the immense amount of traffic traversing these networks
- Extracting meaningful intelligence from a constantly growing sea of data through new applications, viz. Twitter, Facebook, iPad, etc
- The gradual push towards the need for a predictive behavioral analysis
- Lack of product differentiation and transparency between vendors as the LI industry moves towards commoditization, fierce competition, and technological stagnation
- Rise of cloud computing, open source technologies, and increasing power of commodity hardware with potential to replace vendor-specific proprietary black boxes
- Emergence of clusters of COTS GPU-enhanced servers replacing monolithic supercomputers in HPC applications for the defense industry
Consequently, an ideal monitoring system should be centralized and aim for the following:
- Be vendor neutral and designed to accept inputs from any system
- Export data in any documented format to an external system free of vendor constraints
- Source code as a part of knowledge transfer
- API specified to accept inputs and outputs to/from the third party systems
- Be centralized for meeting data processing needs of all security agencies
The vision for such a centralized system should be that of a single, immensely scalable system capable of collecting, storing, and managing up to petabytes of data, which allows for a real-time analysis and extraction of intelligence and information from across a vast number of different data types-voice calls, web pages, mobile call, VoIP, SMS, instant messenger conversations, interactions in virtual worlds, and emails.
But so far in India, we continue to promote deployment of various makes of lawful monitoring solutions selected by our operators, which are not designed to support the above changing demand for handling unstructured content, processing large amount of data or supporting a centralized system. Consequently, in a dynamically changing communication environment, our LEAs continue to face many handicaps in meeting our national security requirements.
|
|
Lawful monitoring is deployed at the edge of operator's network using different technologies as chosen by an operator based on techno-commercial consideration often at the detriment of the security requirements.; | Across all operators' networks based on one technology for security requirements getting primacy |
Multiple means of communication using multiple means for lawful monitoring | Multiple means of communication using one means for lawful monitoring |
| Purchase based on the operator's evaluation of security needs | Purchase based on the needs of the security agencies |
| Captured targeted outputs available to LEAs in different formats | Captured targeted outputs available to LEAs in an uniform format |
| Synchronization of captured outputs across various operators for post-processing and analytics-a difficult task | Synchronization for post-processing and analytics is easy |
| Prone to leakage at many points | Captured outputs are secure at defined location of access |
| Delay in target definition and processing of captured records for timely action | Target definition and processing of captured records is happening in real- time |
| Scaling up and provisioning of new monitoring features is a challenge | Provisioning of new features and scaling up of operations is comparatively easier |
| Monitoring of roaming targets and audit of operator's network for compliance is difficult | Monitoring of roaming targets and audit of operator's networks is taking place in real-time |
| No central administration is possible | Central administration facilitates ease of operation |
| Targets get intercepted only in an area when they are on the operator's network | Targets get intercepted anywhere in the country |
| Lack of confidentiality, since the target definition is done by the operator | Confidentiality is fully maintained, since the target definition is done by LEA who have access to monitoring network |
| Number of LEAs having access is limited by installation as is available with each operator | Number of LEAs having access is totally in hands of administrator of the centralized monitoring platform |
| No audit trail available to LEA for monitoring misuse and this is always dependent on the operator's cooperation | Independent of operator, full audit trail available to monitor any misuse |
| List of IDs blocked from interception, ie, blacklist is at the discretion of each operator | Administrator controls blacklist of IDs that cannot be monitored |
Correlation of CDRs (and LDRs) across operators is difficult and the historical analysis is more manually driven | Correlation of CDR (and LDRs) across multiple operators is easy and the historical analysis is quickly doable at the central server level |
Current Deployment Handicaps
Advantages of proposed centralized deployment vis-Ã -vis the handicaps being faced with currently deployed decentralized lawful monitoring are as provided in the box:
Procurement Process Needs a Radical Change
The current procurement of lawful monitoring is done individually by each operator and each such deployment is approved by the DoT following installation. Since such investment gives no returns, operators end up making purchases that just about passes the defined requirements. This results in LI deployments with disadvantages as stated above.
At the same time, where monitoring center needs to be available with the LEAs, in its absence, its functionalities are often pushed on to the operator for storage and rendering, which at times compromises security. Most LEAs do not possess modern monitoring centers that can decipher-in near real-time-modern means of communications.
In some situations, with the purpose of indigenization and avoiding a tender process of purchase, the government depends on C-DoT to meet its requirements for lawful monitoring, which works at re-inventing the wheel in a manner where their offering is often not benchmarked against any international standards, viz. in case of IP monitoring deployed exclusively by all Indian ISP operators.
Also, all government purchases from the market are done through a public tendering process that lacks secrecy and confidentiality and often results in purchase of equipment and solutions at the lowest price, which do not fully meet the security needs.
To overcome these handicaps, the whole procurement process for lawful monitoring needs to be redefined. First, a roster of vendors should be prepared who have a large experience in providing such lawful monitoring solutions globally. Such vendors should be legally bound to secrecy and confidentiality and thereafter the felt technology based needs for national security should be discussed with them for procurement. Only vendors who can meet the pre-defined requirements of mediation platform with standardized outputs (as per ETSI) for integration with a Centralized Monitoring System (CMS) should be put through a TEC test process and then shortlisted for interaction with the Indian operators.
Tiered Structure
Additionally, the government should set up a 2-tiered structure as stated below:
- With the objective of deployment, uniform mediation platforms for lawful monitoring by each CSP, a National Security Council should be set up, of which each operator's CTO is a nominated member who is responsible for deploying a full compliant mediation solution on his network sourced from any approved vendor as stated above. Each CSP funds the deployment of the mediation platform on its network that provides ETSI standard based outputs to CMS.
- With the objective of deploying and managing all mediated content coming from each CSP, set up a National Security Authority (NSA) for managing CMS, which is funded by the government and will fully manage an immensely scalable system private cloud system capable of collecting, storing, and managing up to petabytes of data. Security agencies should have full access to NSA's services.
Deployment of CMS
Under the above proposed procurement process, we envision the deployment of CMS in 3 tiers in India:
- National Hub: Located centrally at Delhi (with full redundancy at another location in India for disaster recovery) to which all relevant data will be backhauled.
- State Hub: Data collected from each site in each state will be backhauled to the state hub, which in turn will be connected to the national hub via a WAN.
- Site Nodes: Located at each collection point with each operator. These collect data from the operator, perform minimal processing (time stamp, etc) encrypts, and then backhauls data on an eventual consistency model back to the state hub.
Using Vendor Equipment
Use of vendor-proprietary equipment in this model would be limited to the collection devices at some of the operators' site nodes with rest of the system for CMS being on an open platform.
In the next issue, I wish to elaborate on the technical architecture that we should be working towards for enhancing the security of our country.