Blog about Content filtering techniques

Blog had moved to other place

2008-01-16T00:40:00.000-08:00

I had decided to close this blog and move all blogging to the my blog. I'll continue to post entries, and will label it with 'content filtering' tag. Please update your subscriptions

Squid & Google Safe Browsing API

2007-10-08T12:46:00.000-07:00

Had finished today a small utility for implementing support of Google Safe Browsing API in Squid - squid-gsb. Documentation and build instructions you can find on my home page.

New release of OpenFWTK project

2007-09-27T10:07:00.000-07:00

OpenFWTK project had announced release of new major version - 2.0. OpenFWTK - firewall, that has extended features for filtering of application protocols - HTTP, SMTP and other. This project extends ideas of TIS fwtk and keep backward compatibility at eth API level.
System provide ability for development of own application proxies via API, that allows to quick and flexible extend system functionality.

New version of Webwasher

2007-09-25T12:29:00.000-07:00

Secure Computing company had announced release of new version of Webwasher - web-filtering system, providing functions of secure web gateway and hide users from external threats - malware & spyware, spam, etc.
In new version was introduced web caching functions, that allow to decrease access time, and better integrate system into the corporate networks. Besides this, was implemented many improvements in the Trusted Source reputation system, proactive engine and handlers of different formats.

Google continue to attak information security market

2007-08-10T00:40:00.000-07:00

Forbes reports, that Google got approval from the Federal Trade Commission regarding acquisition of the Postini Inc. - company, that implement hosted services in the branch of informational security - transferred data protection (both Web & e-mail), communication compliance, etc.

Article about modern trends in content filtering

2007-07-23T06:46:00.000-07:00

I finish translation of my article about modern trends in content filtering branch (previously available only in Russian). You can find this article here (article also available as pdf).
P.S. May be translation not so good, so please leave your comments here or send them to alexott at gmail dot com

Websense got permit for SurfControl acquisition

2007-07-12T05:55:00.000-07:00

At the begin of the week Websense Inc. got from U.K. office of fair trade permit for acquisition of SurfControl. Permits from the USA commissions was get early.
Merging with SurfControl allow Websense to improve products line, and if merging of existing products will successful, this allow to provide to customer wide range of content filtering products.

New version of SmartFilter

2007-06-20T07:46:00.000-07:00

Secure Computing Corp. had announced new version of SmartFilter - url-filtering product. In new version was integrated TrustedSource technology, that improve quality of checking, by using feedback from installed products, that also integrate this technology.

New product for content-filtering in Web 2.0 world

2007-06-20T04:57:00.000-07:00

Week ago Websense unveils new, sixth version of their data leak prevention product - Websense Content Protection Suite. In new version, was extended functions of data classification, that is very useful for sites, that actively use Web 2.0 technologies - content syndication, dynamic page generation, etc.
By using this product, Websense confirm conclusions from the Gartner's report on Content filtering and data loss prevention. Acquisition of Port Authority made a positive influence on product line of Websense.

Google will active player?

2007-06-19T03:43:00.000-07:00

Today Google had announced the Safe Browsing API, that allow external developers to use database of phishing & malware resources. As example, they show plugin for Firefox web-browser, that check addresses against this database.
Such active deployment of their technologies, could make Google very serious player on the market of content-filtering and information security. Companies, that sell url categorization databases, may loss part of their income.
It will be interesting to implement plugin, that will use this API in open-source projects, such as, Squid or Dansguardian.

Google start to work in information security branch

2007-05-30T04:16:00.000-07:00

Several days ago Google had acquired a startup, that provide solution for malware detection. To reflect works in this branch, was opened a new blog - Google Online Security Blog..
Entering of web-search companies to the market of information security, could has a big impact for the development of new content filtering systems - from using collected information to form a URLs databases and/or databases for online data categorization, to development of new methods of data filtering.

New image spam method

2007-05-08T03:57:00.000-07:00

Secure Computing point in article, that spammers has a new method for deploying spam - instead of putting image directly in mail message, their upload images to the well-known photo hosting sites, and put in mail message only link to the spam picture.
This technique is too hard to detect, and currently nobody can to solve this problem. Only way to fight with this approach - detect networks of compromised computers, and block all messages from them

new product for data loss prevention

2007-05-08T01:28:00.000-07:00

On the market of data loss prevention appear a new product fromMcAfee Inc. - McAfee Data Loss Prevention.
I need to mentionm that this product control both network data (email,P2P, IM, Skype, HTTP, HTTPS, FTP) - with centralized proxy-server, and desktop data (Wi-Fi, USB, CD, DVD, printers, fax and removable devices) - with special desktop agent. Agents and proxy server could be installed separately, depending on the organization needs.
Main platform for the products - Windows. All gathered data are stored in the MS SQL Server.

Acquitions is continue?

2007-04-27T07:36:00.000-07:00

Had found in news, that Websense had announced, that it intended to acquire SurfControl.
Number of players at the content security market continue to decrease - Who will acquired next?

About information leakage prevention

2007-04-24T01:41:00.000-07:00

The Gartner company had published very interesting report about market of content monitoring and filtering/data loss prevention products. Gartner point, that currently this market is small, but it should grow up to $120-150 million in current year.
Currently in this branch (by the version of Gartner), there are 3 leading companies - Websense (after acquiring of PortAuthority), Vontu and Vericept. Many providers of content filtering solutions (Clearswift, Secure Computing, etc.) was excluded from the analysis, as their products doesn't has a required functions - absence of data collection and static analysis, limited capabilities for the filtering of outgoing traffic, no common subsystem for defining and controlling filtering policies.
It's interesting to mention, that many of things, described in this report - collecting and classification of information, linguistic analysis, etc, already was described by me as the things, that will needed for future development of the content filtering systems. I'll continue analysis of data, and will write about the results in this blog.

about notes in blog

2007-04-24T01:37:00.000-07:00

I want to mention, that i also has some notes about content filtering in my personal blog. They are marked with Content Filtering label. But i'll try in future may avaialable only in this journal, or provide a copy of the notes

My article about content filtering trends & techniques

2007-03-16T00:08:00.000-07:00

Today i had uploaded to site my article (in russian) about content filtering trends & techniques, that was published in October 2006 JetInfo Magazine. Article is available here. I hope, that i make translation of article to English in near future.
I had also uploaded earlier article about web-filtering and our product for it, that also was published at JetInfo, and available from their site.

New Acquisition of Cisco

2007-02-21T23:43:00.000-08:00

Yesterday Cisco had reported, that it'll acquire a company Reactivity, that sell a XML-oriented appliances. Interesting thing is that, these solutions could play several roles - xml-data filtering and transformation, provide additional shield for xml-based services, etc.
With this acquisition, Cisco will open for itself a new rapidly grown market of EII- & SOA-oriented products.

New version of WebWasher

2007-01-30T07:36:00.001-08:00

Secure Computing had announced a new version of WebWasher product - 6.5. It also released a new version of Sidewinder - 7.0.
Both product included Real-time Reputation Scores, that allow users of both products to filter not only by categories of sites, but also with reputation rank, that is updated in real-time, and allow to detect and propagate knowledge about sites, that has malicious code

i'm back ;-)

2007-01-04T02:01:00.000-08:00

After long time offline, due my moving to Germany, i'm now online and shall continue to post a news about content filtering

cisco buy a e-mail filtering firm

2007-01-04T01:59:00.000-08:00

News reports that Cisco had buy a Ironport, a company that work in branch of e-mail filtering and spyware controlling.

new blog about content filtering

2006-11-08T01:09:00.000-08:00

I had opened a new blog about content filtering. Now it is in Russian. You can found it here.

stopping the image spam

2006-11-07T23:20:00.000-08:00

Tumbleweed had announced yesterday a new technology to fight the spam, that going as an image. They call this technology "Adaptive Image Filtering", and use combination of wavelet transformation with fuzzy matching.
This is looks interesting, but i need to test this technology to say about results

web-filtering in web 2.0 env

2006-10-09T06:44:00.000-07:00

Introduction of Web 2.0 technologies - Ajax, etc. is headache for developers of Web-filtering products. Information in such web applications smeared between different connections, and sometimes, we can't analyze data properly.
Now i think about developing a new mechanisms for filtering of this applications (many of them are web-mail, chats, etc., that could lead to information leakage). Currently i hold in head blured image of possible approach, but still doesn't have a clear idea.

information interception

2006-09-27T01:31:00.000-07:00

last time, many russian customers (and also customers from other countries) ask us about solutions to intercept internet traffic (web, mail, im, voip). Some of them also want to use this system together with usual phones interception systems.
As i know, in Russia exists some systems, that implement this in some way. But no one do interception of all types of traffic. And we, also trying to play in this game - we has several approaches to this, all based on out MailBoss product, combined with different sniffers and dissectors for various protocols, that produce mail like messages and send them into MailBoss, that do text & datatype detection, and process it with different filtering rules.
But interception of full traffic, has different disadvantages - we need to store anormal amount in information, and we can't use usual relational database to do this. So, currently i thinking about implementing something like data storage, used by web search systems.

Content filtering vendors consolidation?

2006-09-19T05:59:00.000-07:00

Looking back to the summer's time, i think that security products vendors are started big consolidation process. What we'll have from this process - improved products, that cover all aspects of content-filtering and security, or not? Or they just to trying increase their market share?
I hope, that they go first way, not second ;-)
But i also hope, that not only market share will main decision for customers, and also will select products by they functionality

WebBoss again

2006-09-19T05:21:00.000-07:00

We had started development of a new version of our WebBoss (aka SKVT) software. In this version we'll provide following features:
- conditions on HTTP commands, quota's and much more ;-)
- integration with ISS & NetStar URL databases
- authentification of users by IP & MAC addresses
- storing POST's data in database archive
- replace objects with given data
- new web-based interface with heavy use of Ajax
We has too little time to implement this (until the end of the year), but we'll do this!

IBM acquires ISS!

2006-08-24T03:43:00.000-07:00

Got news that IBM buys Internet Security Systems - this is very big strike to the another computer security companies. With IBM's channels ISS will move their products to more number of customers

content filtering of IM

2006-08-23T05:28:00.000-07:00

Alladin had developed a new function for their eSafe product - control of IM file transfer. This is very handy feature for companies, who are using IM for their work

DansGuardian. Code quality in open source projects

2006-08-23T04:00:00.000-07:00

Today i had look again to the dansguardian code, and had found, that they still converted upper/lowercase with arifmetics operation - they doesn't use locale information, provided by using standard tolower/toupper functions. This wonder me, as i sent my patcht to fix this about 2 years ago, when i planned to use DG in some of open source projects.
As i know - both DG developers working in SmoothWall, and if they programm CorporateGuardian in style of DansGuardian, i wonder how it works!

New version of MailBoss

2006-08-17T04:11:00.000-07:00

We had released new version of our mail-filtering software released. Official press release is available here. The fourth version of MailBoss is available at ftp://ftp.jet.msk.su/pub/dozor/dozor-4.0/
Now we have only russian documentation, but in few days we'll released an english version of documentation

ahhh, another acquision

2006-07-28T05:21:00.000-07:00

In start of July SonicWall had acquered MailFrontier company with their e-mail security products

continue to read google alerts

2006-07-28T04:50:00.000-07:00

found an article, in that says, that content-filtering products must make real-time analysis, and don't use pre-defined URL databases. We always use real-time analysis in our products (first product was offered to customers 6 years ago).
There are many approaches to the real-time analysis. Some of them use statistical approach, some of them use genetic analysis technics. Statistical techniques are simple to implement, but powerfull, while genetic is more precise, but harder to implement.

another acquision

2006-07-28T04:32:00.000-07:00

Had found other news about acquision in security market - SurfControl had buyed BlackSpider -- provider of managed security services.
Interesting news, but who will next? How many companies will acquered by market leaders?

acquisions in security market

2006-07-28T04:11:00.000-07:00

Only today, i had found news, that Secure Computing acquired CipherTrust. This acquision will bring Secure Computing to the higher level of secure information management. Secure Computing had acquired Cyberguard about year ago, with it good web-filtering products. And now, Secure Computing got good mail-filtering products.
I'll need to evaluate their products in near future, to see how they compose together products from different vendors

new version of WebBoss almost ready

2006-07-19T05:55:00.000-07:00

I'm pleased to announce, that we almost ready to release a new minor version of our web-filtering product WebBoss. In new version we had increase speed of data processing and robustness of system

URL's categorisation

2006-07-17T01:07:00.000-07:00

Currently our company is on finish line in process of agreement on inclusion of ISS (Cobion) URL database in our web-filtering product. ISS provide good coverage of Internet, working in many languages. One problem is - we need to integrate ISS's categories into our category list.
There are no standard categories, that will be supported by all URL's categories vendors, so i need to map ISS categories into our categories. And there are many such things.
In near future i'll integrate URL list from another vendor, and also will write mapping between their and our categories.

2006-06-27T00:48:00.000-07:00

Long time ago, when i discussed with Kaspersky's staff usage of Bayesian approach to catch spam, they spoke that it's inappropriate to use, and "one good thing to use" = usage of external spam databases.
But in last version of Kaspersky Internet Security, they announced, that they use new technology to spam catching - based on the Bayesian approach ;-)

web filtering with ICAP

2006-06-26T23:25:00.000-07:00

Some time ago i had evaluated possibility of ICAP-server implementation in our Web filtering product. It's good idea to split into the communication & filtering parts, but there are some things, that does not allow to effectively implement ICAP-based solutions:

ICAP-client must be smart enougth to make some content filtering tasks before forwarding to ICAP-server, file type detection, for example
Sometimes, during filtering, filtering engine must have more information, that does not provided by ICAP-client
To provide URL-filtering service, connection to ICAP-server is overhead

Thus, usage of ICAP is possible only in environment, where content-filtering is not primary task, and where is good infrastructure of ICAP-enabled clients - Cisco Cache Engine, for example

Content security for Kids in Australia

2006-06-23T03:53:00.000-07:00

Goverment of Australia wants to provide free content-filtering products for home computers, to provide filtering of content for kids (pornography-blocking). This news comes from News.com.
Some ISP's can provide such content filtering as a service.
Currently we also has negotiations with one biggest russian telecommunications provider, to offer content-filtering as a service for schools, companies & goverment organisations

forget to introduce myself

2006-06-19T10:01:00.001-07:00

I forgot to introduce myself after creating this blog.
I'm working in the branch of content filtering and computer security for a 5 years. Currently i hold position of Head of software development department, that develop and supports 2 content filtering products - MailBoss (mail filtering) and WebBoss (web protocols filtering).
We make research in branches, tightly coupled with content filtering, such as - data format detection, language and encoding detection for the text fragments, information categorisation.
So, i hope, that my blog will interesting for you! Until the next post ;-)

voip filtering

2006-06-16T03:37:00.000-07:00

Currently we looks for solutions for VoIP detection, capturing and analyse. We had found open sourced Oreka, which can detect and capture VoIP stream. But main task - how we can analyse voice, extract phrases from it, and get normal text to analyse?
Voice (also as image) recognition is interesting theme, but we need to have there are many ressources to model and implement this task.
Is anybody knows solutions for voice recognition?

about text filtering

2006-06-16T01:43:00.000-07:00

There are several approaches to the text filtering:
- searching of concrete words in text files - it is very simple to implement, and quick for processor;
- searching for regexps - also simple, when using external libraries, but more slow than simple text search;
- there is also another approach to the getting text category - using combination of words with weights. Using positive and negative weights values, we can select data, that match given category. This method is harder to implement, but it still simple. But, exist one problem - how to calculate weights for given words? Manual update of weights and words is hard problem, so it method is not good for use;
- another approach to the getting text category is in using something like Bayesian statistical text classification - we can use automatic text analyzing tools, by comparing texts in several categories and extracting needed information - words, weights, etc. This methid is slower, than others, but it provide good results, together with automatical informatiion extraction.

In our products we use some of these methods to classify texts - users can select which method to use, depending on tasks, that they want to do.

about this blog

2006-06-13T04:29:00.000-07:00

I had posted different things about content filtering in my personal blog, but now i decide to move all content-filtering related things in dedicated blog.
So, i pleased to present you new blog about content-filtering techniques. I'll write about content filtering of e-mail, web, etc. About technical and moral aspects of this. I'll try to make posts frequently.
You're welcome!