Makemedia Smart Q

for Various Clients

  • Process business specific tasks quickly
  • Manage hidden business work flows and processes
  • Process and categorise large volumes of data

The Makemedia Smart Q is a configurable, scalable and robust task queuing system with added intelligence. It allows clients to process their business specific tasks quickly and efficiently. With a list of priorities provided by the client, the Smart Q provides an engine for the efficient running of different tasks with specific responsibilities.

The Makemedia Smart Q is a configurable, scalable and robust task queuing system with added intelligence. It allows clients to process their business specific tasks quickly and efficiently. With a list of priorities provided by the client, the Smart Q provides an engine for the efficient running of different tasks with specific responsibilities.

Some of the key business challenges we have used the Smart Q to address are:

  • Complex legacy processes
  • Manual and programatic tasks
  • Managing hidden business work flows and processes
  • Poor performance of legacy data processing and document generation
  • Unscalable and tightly coupled technology that prevents product innovation
  • Inefficient systems with large support overhead

Built upon RabbitMQ the Makemedia Smart Q adds an additional layer of intelligence to out of the box queue messaging technology. The Smart Q requires little knowledge of the business’s changing and evolving priorities. It is not influenced by what the tasks are trying to achieve but just needs a skeleton definition of what steps each task should be running, and which steps are dependent on the completion of the others. The logic in the steps themselves is independent from the Smart Q engine and therefore if the business’s priorities or workflows change, you simply change the Task Definition.

The Smart Q can be set up using both cloud based hosting services, such as AWS and Azure, or on more traditional dedicated or hosted servers supporting .Net.

The diagram below provides a simple architectural view of how the Smart Q and business process tasks interact. A task could be anything from an API call, a PDF conversion step, email generation or bespoke data processing e.g. Categorisation or Alerting.

SmartQ (1)

The Smart Q is based on a distributed and service oriented architecture, therefore enabling optimum performance. Load balanced servers and the ability to apply a configurable number of processes per task can increase the performance of the engine with minimal effort. The service oriented architecture means that, with nominal development time, the business’s existing web services can be used in conjunction with the Smart Q engine to perform the business specific tasks.

Elasticsearch provides the storage for the queue process which means that the Kibana Dashboard (that comes as part of the Elastic technical suite) can be used to visualise the status of tasks in the queue and see performance metrics. Elasticsearch and the Smart Q logging processes provide an extremely resilient method of handling any errors, or blocked tasks which can be viewed easily in the dashboard. Surfacing logs through the Kibana Dashboard allows for rapid troubleshooting and items can be re-sent to the SmartQ.

Capture

How and Where is it Used?

Makemedia have worked in partnership with our clients to integrate the Smart Q into business processes covering diverse domains and scenarios.

Processing and Categorising Large Volumes of Data

One of these clients is a leader in providing political intelligence products and services. The Smart Q allows them to process newly sourced political information that has been emailed or collected from the web and to then summarise and tailor it to their clients needs as quickly and efficiently as possible.  For every new piece of content a new task is sent to the Smart Q to be processed. The content is sent to different business services that each perform a specific task. First it is indexed and categorised against their taxonomy, then it is matched to their client’s particular interests, and finally it is grouped and emailed out to clients according to their alert criteria. The SmartQ deals with up to 100,000 new items of political content per day.

Creating Documents with different workflows and dynamic content

Another client making use of the Makemedia’s Smart Q, deals in creating environmental reports for homebuyers, consultants and surveyors. In order to create a fully extensive homebuyers report, for example, a vast amount of spatial and environmental data needs to be processed and retrieved from different sources. Customer data, base maps, flood data and risk calculation are all provided by different business services. The SmartQ helps with this as it determines the priority and order of services that are called and the number of processes needed for each data service. Some services may return more data and take longer than others. Therefore in order to prevent delays around the running of these steps, we can scale up those services. It is this intelligence and ease of configuration that separates the Makemedia Smart Q from other out of the box queue messaging systems.

The power of the Smart Q is in its flexibility and its application. Even if you have a number of disparate systems or manual steps in your workflow to automate, the Smart Q has the power and flexibility to address a wide variety of business challenges.

Want to explore how we could bring success to your digital product? Learn how we work, or get in touch with our team of experts.

Back to top