By: Avi Weiss, April 25, 2015
Telecom News
Exposure of the innovative connection between cloud application developers and advertisers through the cloud. How do real-time responses to a million requests per minute are done and how over a billion requests are handled per day by smartphone users around the world?
"When the user is exposed to an ad, be it video or content, behind the scenes there is a real-time trade of advertisements," says Gal Aviv, Director of R&D team, Inneractive." A stock exchange of ads exists with a competition over presence of advertisement areas that users are exposed to. The value of advertising space is dynamic and constantly changing between all end users. The user does not feel this since the whole process takes a fraction of a second. "
I had an exclusive interview with Gal Aviv, to hear how they serve over a million requests for ads a minute, from all over the world, around the clock and throughout the year, and what is the role of AWS (Amazon Web Services) in the fast and enormous process that concerns all Internet users.
Question: What is your job in Inneractive?
Gal Aviv: "I came to the company after serving in the Israel Air Force as a development engineer. Today I manage the main group engaged in R&D in the company core issues, which are: Big-Data, Real-time services, DevOps (a new cloud infrastructure development methodology).
Inneractive is a private company, operating for 8 years through an R&D center in Israel and offices in California and New York, with total of 5 offices around the world. We have 70 employees with very nice growth, also in the number of R&D employees. "
Question: Who are your customers?
Answer: "We are in the middle between several types of customers: end customers, who are everywhere, especially in their smartphones, advertisers and apps developers who are our main customers, wanting to generate money by selling advertising space on their applications. Inneractive handles all those advertising spaces by connecting advertisers to specific advertising space, in order to generate the highest price possible for the application developer.
There are quite a few players "in our space", but this competition is not all simple as it looks from the outside. There are many layers of data handling, and we are in a central and leading position, because we know how to respond quickly and handle enormous amounts of data.
We receive to our systems 24 hours a day, over a million requests per minute, from all over the world. We serve over 1.5 billion requests for advertising allocations per day. This is the pace we deal with. "
Question: What are the most desirable advertisements today?
Answer: "the commercials most requested are in video modes advertisements. Almost every video you watch on your smartphone, tablet or computer, includes in fact some advertising.
Our customers are interested in maximizing their revenue. In fact, behind the scenes of the advertising business a market-stock exchange exists, we at Inneractive are "smart traders" in this market. We do this by hundreds to thousands of percent better than what a portal owner or application developer can do alone.
Our customers, who are large developers and application development houses, can sell advertising ''door to door", but Inneractive can offer them a contacts with 120 of demand sources / request, and those are working with thousands of advertising agencies creating their aggregation. This way, we cover most of the world advertisers.
As I mentioned, Inneractive knows how to serve over a million requests for advertisements per minute, in this very minute a "smart bid" is done. The system checks for the optimal result for the customer, and returns the most advantageous ad. This is done in 200- 250 milliseconds. The user does not feel this activity, which takes place in an instant behind the scenes ".
Question: What is your profit from each ad?
Answer: "There is no fixed price. Any deal is signed with a client, following negotiations for its terms. Each client has different terms. In some of the cases, pricing is done dynamically, in real time, reflecting demand and supply".
Question: How do you do the matching between advertising space and the ads so quickly?
Answer: "Our uniqueness is in using the fast and elastic services over AWS. This is a process that occurs billions of times a day, and this is why a partner like AWS is so important, providing us with pay-per-use only, unlimited computing and storage resources without having to purchase and operate them. We know how to make parallel calculations for a large number of advertisers.
"There exist constant and quite strict protocols in the advertising industry, both in terms of response time and all the advertising rules. We know how to exercise several protocols in parallel, so we know to respond to any request and make the best and fast possible decision. We provide a response within about 200 milliseconds even before the user could feel it and clicked for example on the video Play he wants to watch. "
Question: How do you execute such a fast process?
Answer: "This is definitely a challenge we are dealing with. We are facing a lot of advertising partners, so we had to develop a system with very high level of performance, a system that knows how to use new models of software development, based on Actors Model. This gives an abstraction layer to manage parallelism. To achieve this we write and develop in a language called Scala with a tool called Akka.
The main problem we face is not only parallelism. It is the need to go up and down in resources used very fast. Take for example a popular sport event. This event has commercial breaks, and then almost all viewers open their smartphone looking for interesting things related to the game or such that interest them. At that moment, the systems get sometimes 3 times requests for advertising. After a few minutes, the game continues and the demand gets back to normal.
No system in the world has the capacity to cope with such fast changes in zero time. You have to understand that as soon as the demand jumps 3 times, we have to respond as quickly. No end user is willing to wait for a response and he does not care about network congestions.
This is where Amazon's cloud comes in. Because Amazon is deployed throughout the world, we want to be physically close to all the major players in the advertising world, as they sit as well in Amazon's infrastructure. If you are in the same "servers' farm" with advertisers, your reaction time is much faster.
So this is how Amazon cloud environment allows us to identify loads and increase or decrease the resources used. In reality, we are in a range of between 100 running servers at the same time, jumping to 300 to 400 servers running simultaneously. This change is done in seconds.
These changes are happening all over the world almost in front of all of our customers and in front of our traffic aggregators. Some of these sites have put our code, and this can be anywhere in the world. These sites collect amounts of information and this is a recipe for permanent changes in the volume of requests.
Those changes in demand in real time are amazing.
Therefore, it is convenient for us to sit on Amazon infrastructure. There is no logic to build and manage "server farms" all over the world, as this is a problem which is difficult to deal with. This is a huge logistical problem and a serious problem for us too being an R&D unit
We are constantly growing, so we have no resources construction to deal with, leasing or management of data centers. Amazon provides us with everything, including all the maintenance needed. For example, all our developers control the resources they need any moment. All is dynamic ".
Question: Do you have more products and solutions?
Answer: "Absolutely. We have solutions for the world of Big Data. We produce millions of transactions every minute and billions of transactions every day. We can learn from it a lot.
We carry on these huge amounts of data analysis in several directions and several uses:
A. A real time analysis to improve our performance. These help us to constantly improve.
B. Analysis allowing us to present to our clients - application developers and their performance in real time. They know at any time their performance against the market. This includes analyses of what their users like and in what they are better or not as good. We work with thousands of app developers, and also with hundreds of publishers and portals, implementing our code. They transfer enormous amounts of information and get from us their performance analysis.
C. We operate extensively, analyzing to neutralize the issue of fraud. There are smart sophistication levels here and we have a system that can counteract and neutralize fraud and fake traffic. In addition, as a company policy from its inception, we do not touch content such as porn or gambling. So we neutralize such traffic as well, as that is a part of our DNA since we were established.
D. We trade our information with those who want to buy information under legal restrictions in each country. It is not a simple process as we have to comply with many standards of database handling. Since we have huge amounts of data and customers who want to purchase information, wanting only specific pieces of it, we have to know what to provide them with. Sometimes, parts of this information are confidential, whether because of local laws or because of privacy and agreements we made with our advertisers and customers. All that has to be taken into account when presenting the analyses.
We could not do all our Big Data activity without a huge cloud infrastructure. With the loads that we have, with so many elastic components in our system, there is always increase or decrease in the information components. Any information element can grow in seconds several times of its original size.
We tested the performance of AWS and we chose it and this created a very tight collaborative process between us. "
Question: How this cooperation with AWS is reflected?
Answer: "We actually have become beta testers for Amazon. We are exploring new technologies from Amazon, not yet in the market. Thus, we can test advanced products of Amazon and can give them feedback while they are still in the development process of these products.
This collaboration is so close that Amazon engineers are coming from the US to Israel to discuss with us our comments. It happens all the time because they want to constantly improve their performance and we give them a very good feedback.
It is amazing, how a huge global company such as Amazon with revenues of $ 22.7 billion in the first quarter of 2015, of which 5 billion come from AWS, is attentive to individual customers, small and far away like us. This is a true collaboration. Thus, our feedback directly affects the operation of Amazon cloud and its capabilities all over the world. "
Question: How do you find developers for Inneractive?
Answer: "We currently employ 38 developers, which we find ourselves, using the "referral program". We are actually looking for people with tweezers. Nurturing our engineers and developers is a task, in which we invest a lot of thought, with the ability to cross-fertilization between the people all the time.
To keep these good engineers is not a simple challenge. We are a technology company, so we give our engineers constant challenges. Thus our engineers have constantly motivation to develop and achieve technologies and solutions, which they achieve thanks to the mutual enrichment and team work. "
Q: Where are you developing towards?
Answer: "the advertising world today includes two key growth drivers: video publishing and advertising for Native Ads. The other issue is displaying advertising content within the content that already exists. There are those who call it 'marketing content. It acts like a regular content but behind, there is an advertising content.
We in Inneractive know and are able to handle any configuration of existing or future advertising. There are differences between the consumption of video content and reading this article. Our system is able to cope with the differences between these types of consumption.
In Video there are very precise measurement systems today. It is known, for example, not only how many views there were per video, but whether the viewer watched the video to its end and where he stopped to watch the video.
Therefore, we are able to price the use of any video according to how it is used. Thus, those who are watching the video to the end are more exposed and one should get more revenue from them.
In the field of Native Ads, we know how to embed the ad within the content everywhere so not to hurt the user experience reading the content. Today, we are able to combine all the options together, including video, right into the content.
In addition, we collect lots of information about users subject to existing privacy laws in each country. If we know who the user is, we'll stream him content, there is a high probability he would get into and read. This action takes a second particle. Less than 200 Millie seconds and we know what content to display to the user.
We are working on providing service by world regions so we can navigate the trade in advertisements by regions in the world. We understand how global traffic is working and can give attention to each area with high levels of durability, as Amazon is capable of providing demand information by regions, which are constantly changing over the day and days of the week.
The world is going now towards open source and we see this as the future of software. We have our own architecture, built on open source, adjusted to our capabilities with many internal developments to perform Big Data tasks. It's part of our strength and we want to make open source implementations for any development of our customers need.
So, we do everything needed so that and we are not dependent on others when it comes to our core business. We will continue and develop these directions as we believe that that this is the future"
Question: What is your professional dream?
Gal Aviv: "The dream is to lead the dynamic ads domain in the web. We are in a good position in a very diverse and very competitive area.
Today we are faced with challenges that the world's largest companies are facing, including world-class real-time developments of Big Data applications. My desire is that we can provide information in real-time in the best way possible for every customer all over the world.".
