If you want to see what is coming up in the market of consumer-technology (PC, mobile and tablet), then NVIDIA can tell you the most. The company is very flexible, and shows time after time it really knows in which markets is currently operates and can enter. I sometimes strongly disagree with their marketing, but watch them closely as they are in the most important markets to define the near future in: PCs, Mobile/Tablet and HPC.

PCs

Mobile

Where CUDA and where OpenCL?

Read more

The Khronos Group gave some talks on their technologies in Shanghai China on the 17th of March 2012. Neil Trevett did some interesting remarks on the position of NVidia on OpenCL I would like to share with you. Neil Trevett is both an important member of Khronos and employee of NVidia. To be more precise, he is the Vice President Mobile Content of NVidia and the president of Khronos. I think we can take his comments serious, but we must be very careful as these are mixed with his personal opinions.

Regular readers of the blog have seen I am not enthusiastic at all about NVidia’s marketing, but am a big fan of their hardware. And exactly I am very positive they are bold enough in the industry to position themselves very well with the fast-changing markets of the upcoming years. Having said that, let’s go to the quotes.

All quotes were from this video. Best you can do is to start at 41:50 till 45:35.

At 44:05 he states: “In the mobile I think space CUDA is unlikely to be widely adopted“, and explains: “A party API in the mobile industry doesn’t really meet market needs“. Then continues with his vision on OpenCL: “I think OpenCL in the mobile is going to be fundamental to bring parallel computation to mobile devices” and then “and into the web through WebCL“.

Also interesting at 44:55: “In the end NVidia doesn’t really mind which API is used, CUDA or OpenCL. As long as you are get to use great GPUs“. He ends with a smile, as “great GPUs” refers to NVidia’s of course.

At 45:10 he puts NVidia’s plans on HPC, before getting back to : “NVidia is going to support both [CUDA and OpenCL] in HPC. In Mobile it’s going to be all OpenCL“.

At 45:23 he repeats his statements: “In the mobile space I expect OpenCL to be the primary tool“.

What does it mean?

The main question in the viral video “double rainbow” is a good one. Specially because it is always a hard question.

For me these statements mean that at NVidia they have been thinking it through very well. I like that for two reasons. First that they are long term thinkers, second that they get the competition at the ARM-market on hardware-capacities and pricing.

Maybe not emphasised enough, but the market is going to change a lot. In short: the “middle market” (PC desktops and laptops) are going to disappear and/or being replaced. NVidia is anticipating a lot by focusing on two areas at both sides: HPC and mobile.

HPC

Know Cobol and Fortran? Those are ancient languages that is still used a lot in HPC. I don’t know who’s law it is, but “The more security they want it, the more ancient the methods”. Another example of an industry that is prey of that law is the space-exploration – “better to have fully tested code than anything else”. CUDA would be a very welcome add to these industries, and they know it.

NVidia mostly aims for the (incredibly growing) cloud-part of the future world. It’s most direct competitor: Intel.

Mobile & ARM

NVidia sees that the faster they get to a standard, the faster ARM gets accepted as the next X86. The slower AMD moves, the bigger the market share. Also the “old world” of ARM-companies are not used to the ways of the consumers very well knows in the X86 world – ARM vendors are mostly not directly tied to consumers, but have one (or more) vendors between consumers and themselves. So NVidia has all the space to get big as they have all the stage for their own marketing.

If NVidia is backing OpenCL, then this could give a good push in the right direction to get one standard for compute on mobile devices.

Desktop & Laptop

While AMD and Intel go hybrid with X86, NVidia keeps making discrete GPUs. The result: losing market share each quarter. But if you look at the decreasing sales of these devices, the only growth is by harsher competition. I don’t think it is a bad choice at all.

The 2012 GPUs are split in two groups: gaming and HPC. The differences between these two will give more answers on the path NVidia takes.

Also note that Trevett did not say anything about computers – neither ARM nor X86.

OpenCL in the browser

He was very explicit on WebCL as a standard for computing on the HTML5 web. It is still working draft, but things must go really strange if this standard will be replaced by another standard, such as Intel’s RiverTrail. There are no similar competing products based on CUDA, DirectCompute or RenderScript.

Perspective

Neil Trevett said that a standard in the mobile industry is very important. To get quick acceptance of ARM as replacement for X86, standards (such as OpenCL) would need to get in place soon. NVidia is ready for a quickly shifting market, so some quotes could be seen differently in that light. Say demands for ARM-based computers will raise after the release of Windows 8, and games recompiled on ARM get released, then NVidia will have a better quarter than Intel or AMD for sure.

As these games-on-ARM and other software that needs processing power must start using OpenCL, StreamComputing does not mind.

Now that smartphones get more powerful and internet makes it possible to have all functionality and documents with you anywhere, the computer needs to be reinvented. You see all big IT-companies searching for how that can be, from Windows Metro to complete docking stations to replace the desktop by your phone. A turbulent market.

One of the new products are USB-stick sized computers. Stick them into a TV or monitor, zap in your code and you have your personal working environment. You never need to carry laptops to your hotel-room or conference, as long as a screen is available – any screen.

There are several USB-computers entering the market, but I wanted to introduce you to two. Both of these see a future in a strong processor in a portable device, and both do not have a real product with these strong processors. But you can expect that in 2013 you can have a device that can do very fast parallel processing to have a smooth Photoshop experience… at your key-ring.

FXI Tech Lollipop

Just recently a company called FXI technologies had a presentation in Japan and showed their roadmap containing OpenCL-capable USB-computers. Of course we picked that up:

It has the A15 processor, wifi, bluetooth and USB 3.0. It is not very clear which GPU they use, but very possible it will be the ARM Mali T658.

NVIDIA Tegra-stick

According to Tom’s Hardware: “Nvidia’s filing outlines the idea of packing a Tegra processor, Flash storage, RAM, at least one USB port, and other interface types such as a parallel port, a serial port, IEEE 1394 (Firewire), VGA, HDMI, S-Video, AV, DVI, LAN and WiFi into a package that is just 40-60 mm long, 10-20 mm wide and 5-10 mm thick. The feature set would be comparable to a entry-level computer system with limited storage and cloud connectivity”. So very comparable with FXI Tech’s product, except it has a Tegra-GPU.

NVidia chose not to go for the hybrid X86-based CPU+GPU for desktops and laptops as AMD and Intel did, but to focus on mobile and on (HPC) servers.

Hardware potential in the coming years

In 2015 memory-sizes, processing power per Watt and storage-capabilities will be at least 4 times of what we have now. When OpenCL (and alike techniques) are used by the masses, this can even be more – and that is where these two products bet on. Think 30+GB memory, many-core hybrid processors, GBit internet, Dropbox Free starting at 20+GB, etc.

Companies will have a hard(er) time to earn enough money as the market is mostly satisfied with a TV that has a possibility to attach an USB-stick containing somebody’s personal environment. So instead of wanting more stuff, getting rid of stuff is a stronger reason to sped money. A powerful computer starting at 35 euro (like the Raspberry Pi) plugged in an unpersonal screen+keyboard will help that demand.

As for StreamComputing (and customers) it is very important to know where the growth-market is for OpenCL, soon more on this subject.

By exception, another PDF-Monday.

OpenCL vs. OpenMP: A Programmability Debate. The one moment OpenCL and the other mom ent OpenMP produces faster code. From the conclusion: “OpenMP is more productive, while OpenCL is portable for a larger class of devices. Performance-wise, we have found a large variety of ratios between the two solutions, depending on the application, dataset sizes, compilers, and architectures.”

Improving Performance of OpenCL on CPUs. Focusing on how to optimise OpenCL. From the abstract: “First, we present a static analysis and an accompanying optimization to exclude code regions from control-flow to data-flow conversion, which is the commonly used technique to leverage vector instruction sets. Second, we present a novel technique to implement barrier synchronization.”

Variants of Mersenne Twister Suitable for Graphic Processors. Source-code at http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MTGP/

Accelerating the FFTD method using SSE and GPUs. “The Finite-Difference Time-Domain (FDTD) method is a computational technique for modelling the behaviour of electromagnetic waves in 3D space”. This is a project-plan, but describes the theories pretty well.

Auto-tuning a High-Level Language Targeted to GPU Codes. Auto-tuning is (according to us) a very important field in OpenCL/GPGPU the coming years. This article describes HMPP, a language which uses pragmas, and how auto-tuning can help to maximise performance.

Intel SPMD Program Compiler: A SPMD Compiler for High-Performance CPU Programming. If you followed and liked NVidia-vs-Intel on “speed-up for free”, then you should read this.

Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing. Various benchmarks pop up with GPGPU-support. This is one of them for CUDA-only, (indirectly) funded by Microsoft, Intel and NVidia, but open source. http://impact.crhc.illinois.edu/parboil.php

Dynamic Data Structures for Taskgraph Scheduling Policies with Applications in OpenCL Accelerators. This study is about “stochastic scheduling with precedencies”. Also interesting is that it is done in cooperation with ARM.

A Domain-Specific Approach To Heterogeneous Parallelism. From the abstract: “We propose leveraging domain-specific languages (DSLs) to map high-level application code to heterogeneous devices. To demonstrate the potential of this approach we present OptiML, a DSL for machine learning”.

Implementing FPGA Design with the OpenCL Standard. The official whitepaper of Altera for their upcoming product.

Towards A High-Level Approach for Programming Distributed Systems with GPUs. Introducing SkelCL, a higher level language, and dOpenCL, OpenCL for a distributed systems. No source or binaries, but the presentation describes their approach well to be able to compare with other solutions.

A Comparative Study of Parallel Algorithms for the Girth Problem. A research that compared CUDA against OpenMP with all the results at the last two pages.  The girth of a graph is the length of a shortest cycle contained in the graph.

AESOP: Expressing Concurrency in High-Performance System Software. An alternative to current distributed and GPGPU programming languages. Read to see what could be improved in OpenCL according to these researchers.

Exposing Fine-grained parallelism in Algebraic Multigrid Methods. 30 pages to describe the development of  ”a parallel algebraic multigrid method which exposes substantial fine-grained parallelism in both the construction of the multigrid hierarchy as well as the cycling or solve stage”.

Robust Volume Segmentation using an Abstract Distance Transform. Segmenting noisy CT and MRI data using GPGPU.

Mobile GPUs. A short, nice overview of the current state of ARM-based GPUs.

I hope you liked this overview – help the other readers by telling interesting discoveries in the PDFs in the comments. And if you have been done research in this field, please contact us.

In “Separation of compute, control and transfer” I talked about node-wise programming as a way we should embrace instead of try to replace by loops. In this article I get into loops and discuss for a few types hopw they can be run in a parallel form. Dependency is the big variable in each type: the lower the dependency on previous iterations, the better it can be parallelised. Another one is the known iteration-dimensions known before the loop is started.

The more you think about it, the more you find out that a loop is not a loop.

Loop types

The following loops are the ones I thought of. I did not find a collection of loop types, so they might be incomplete. You’ll find at many places that “some iterative problems let them better be described in while-loops, while others in for-loops”, not the exact definition of the various iterative problems. Please let me know if you know research-articles of books, which describe this problem-field in detail.

Notation-style is from Matlab/Octave.

Independent, fixed dimension

This is code very easily run in parallel.

for i=1:n
    B[i]=A[i]*2;
end;

The only problem that needs to be solved is that each processor needs to do enough and memory is arranged well. These are the problems tackled in all GPU-programming guides. Any problem that can be described in this form, is a winner. For large enough n, this is the example of code that can be sped up the easiest.

Intermediate answers

Each repetition the intermediate answer is updated, giving a result at the end. For instance the following loop.

k=1;
for i=1:n
    k=k*i;
end;

The result is the faculty of k and is not straight-forward to parallelise. One of the ways would be to split the loop in pieces and bring the results together at the end.

Some loops can be replaced by a single mathematical function (without any loops). That is without hardly any exception the fastest way to get the answer.

Iterative search

Finding the answer, by checking the intermediate-result each iteration. These are loops with intermediate answers, but without the fixed dimensions.

i=0;
while result<10
    i++;
    result=A[i];
end;

If the programmer wanted to have the first element in A which is smaller than 10, then the parallel implementation would be different than if any element smaller than 10 had to be found. To speed things up, decisions are made in such code, we would have done differently assuming there are more cores in the processor. But say the first element needs to be found, striping-like techniques would best be used at the GPU such that each of 100 cores searches at element x+100. So this type of loops can be very dependent on interpretation of the actual goal.

Recursive

Recursive functions are not really loops, but I do want to mention them. Recursive functions are very close to human way of thinking in more complex problems. These loops are very serial.

function recur (a, b)
    if a>0
        recur (a-1, b*2+a)
    end;
end;

These functions can be transformed to a simple function (Mathematical software are a great help here), or unrolled to stack-based loops (see below) or loops with intermedita answers. As a recursive function they cannot be run in parallel.

Stack-based and backtracking

A stack-based loop needs more intermediate answers and keeps them at a stack. An example I like is to solve a maze: the moment a dead end is found, the current trace is marked dead and the loop continues a step back. Another one is parsing text.

A parallel function is made by finding split-points in the search-area such that each branch has the same weight. These loops (most times) don’t have fixed dimensions, so weighting the branches running-time can be tricky.

Controlled

Within the loop there are control-statements (like “if…then”) or statements that alter the variables of the loop. Also while-loops are controlled code, as the result is checked each iteration. See also the iterative search.

k=3;
for i=1:n
    if mod(k,29)==0
        k=3;
        i=i-1;
    else
        k=(k-1)*k;
    end;
    B[i]=A[i]*k;
end;

Altering the loop while running is quite hard to interpret by compilers (both human and machine). So first, clean up the code by finding out how many times the code needs to be repeated. Second decide if it is

(Note that control-statements are expensive on GPUs (especially AMD’s) and should be avoided, if possible – in this case very doable as all possible values of k are known and very limited).

Final words

But if you work with 3D-coordinates, it seems no problem to solve many types of puzzles. And what about that we can get al multiples of 5 up to 100 by adding 5 iteratively, but also by multiplying [1..20] by 5. All can be described in loops, while all are very different problems.

Instead of a double loop, why not the following to stay close to node-wise programming?

rangerun i=0:0.001:pi, j=pi:0.001:2*pi
    result[i,j]=sin(i)*cos(j);
end;

Maybe new keywords will not be the best solution – IDEs could have a wizard to transform loops into range-runs, decreasing guess-work by the compiler. I strongly think we should not mix this no-dependency loops with high-dependency loops, and do try to make the programmer aware what he or she is doing. Iterating, building up to an answer, or running a range of data-computations.

Say you have a device which is extremely good in numerical trigoniometrics (including integrals, transformations, etc to support mainly Fourier transforms) by using massive parallelism. You also have an optimised library which takes care of the transfer to the device and the handling of trigoniometric math.

Then you find out that the strength of your company is not the device alone, but also the powerful and easy-to-use library. You also find out that companies are willing to pay for the library, if it would work with other devices too. From your own helpdesk you hear that most questions are about extending the library with specialised functions. Giving this information, you define new customer groups for device-only and library-only – so just by adopting a standard you can increase revenue. Read below which steps you have to take to adopt OpenCL.

OpenCL as intermediate language

OpenCL could function very well as an intermediate language. It is an open standard (no royalties need to be paid and specifications are freely available), so ownership of the software is not given away while it is plugged in a larger ecosphere. The specifications need to be implemented including compiler – this is quite some work, as you can understand. By aiming at the same audience in version 1 (less support for the non-trigonometric functions), you can separate the library and the device with minimal overhead. This can be done without telling anybody, as it is an open standard. But mentioning you use OpenCL is not allowed, yet.

Claiming the library supports OpenCL

The moment you want to sell your library as a separate product, claim it supports OpenCL and use the logo on your product, you need to sign a form and send it to Apple. That is all. You are responsible yourself it actually supports other devices – not supporting devices well or not being explicit about which devices have been tested, will not do good for your name. This includes testing a lot of devices (and optimising the kernels for them) – I see that most target the well-known ones (recent Intel/AMD CPUs and NVidia/AMD GPUs of 2010 and later).

Claiming the device supports OpenCL

Claiming the device supports OpenCL need to follow a few more steps, the so called Khronos Conformance Process. As you also gain access to any available sample implementations by becoming a member, you might want to do this before implementing the specifications.

First you need to contact Khronos, sign a agreement and pay a fee to enter the process. This is US$ 15 000 for unlimited products per version – so if OpenCL 1.3 or 2.0 comes out, you need to pay the fee again, regardless if you have one or ten devices and the number of tests you do (so in other words you can submit unlimited conformance results for that version). Conformance fees for an API cover any number of product submissions using any version of that API up to the paid API Level (some exceptions can apply). From that moment you are allowed to use the name OpenCL, but not the logo for the device. This is handy if you want to tell you are working on it for marketing purposes. Academic members pay US$ 1500 and Khronos members pay US$ 10 000. You can also become a member of Khronos for US$ 10 000 a year, giving you the right to join a working group and voting rights – but to be clear, you do NOT need to be a member to adopt OpenCL (or any other Khronos technology).

Second you need to run a suite of tests provided by Khronos. If your drivers work and you send back the results, you can fully use logo and trademark (with small disclaimer). After a peer-review of the results with positive results, you can fully use the OpenCL-logo for your device and are listed on the Khronos website.

Final words

What I wanted to show here is what the “open” in OpenCL actually means, while showing how accessible the Khronos group is to new adopters of its technologies. Simply put: anybody can join, if you follow the guidelines and pay for the costs. So anybody can compete in the field, bringing more specialised and faster products.

A while ago I spoke with Patrick Viry, CEO of Ateji. We shared ideas on GPGPU, OpenCL and programming in general. While talking about the strengths of Ateji PX (a Java-like language for parallel programming), he came with a remark which I found important and interesting: separation of transfer.

Separation of focus-areas increase effectiveness but are said to be for experts only. For example the concepts of loops is well-known to programmers, so that seems to be reason enough it should be the starting-position for any goal concerning repetition. Current lower-level GPGPU languages are kernel-host languages and describe what has to be done at one coordinate (or group of coordinates) in the data instead looping over the whole data. What I see is that the idea is getting abandoned in higher level languages instead of turned into a design pattern. I would like to discuss these separations, to see where the focus on higher-level languages on top of these low-level languages could and/or should focus on.

I chose the image of a tree, as you have the food coming through the roots, the transport goes through the trunk and the complex stuff happens in the separate leaves.

Separation of compute

Where in the concept of looping, one can choose to loop over (i, j) of the input-data or (k, l) of the result-data, in OpenCL one is forced to do the computations seen from each (k, l) of the result-data (with some exceptions). An important difference is that you cannot have temporary-answers and use them in the next iteration. If iterations are needed, then data can be computated in several steps.

By forcing the programmer to think this way by separating the single computation from the repetition, the code can be optimised and scaled easier. And exactly this is what is abandoned in higher level GPGPU-languages. You see the low-level languages are all on the left, and most of the higher-level on the right. Exception is ArrayFire, which stays close to the Matlab-concept.

Node-wise (each group of data-elements on which is computed on) has the advantage of scaling, but takes some time to implement by programmers who are used to loops. Functional programming solves a lot, but is not applicable to all kinds of problems – I do like this approach and it could be a good direction. Unlooping is a research-area very well explored, but it is still not optimal if it comes to scaling – that’s why it is still a big research-area.

I think the strength of separating the computation from the rest is undervalued.

Separation of transfer

When using GPUs, but also when reading a file from disk, part of the time that the whole operation takes is transferring data. This needs scheduling. Scheduling data-transfer is the most part of the host-code in OpenCL.

Here is explained how Ateji PX does this explicit transfer-scheduling. The choice for most new higher level languages is to leave it to the compiler to find out, even if explicit transfers could increase time. Please let me know, if it can be done with i.e. OpenACC‘s async.

While OpenCL and CUDA have room to improve this separation, only Ateji PX did not abandon it. Forcing programmers to explicitly defining this, increases the overall speed. If the programmer does not think about where things happen, then next year hardware with low transfer-speeds will perform best – this decreases potential growth of many types of dedicated co-processors such as FPGAs.

Final words

Even if the old ways of programming have shown their scaling limits, most higher level languages which aim to replace OpenCL and CUDA, focus on trying to trust the old paradigms. The good things that CUDA and OpenCL brought is exactly this separation of computation and transfer, and these could help us all entering the multi-core era. CUDA and OpenCL could still use a lot of optimisation such as automatic pinning of memory, optimisation of simple kernels without coalescing and manual caching, etc. I invite language and compiler designers to focus on that, instead of making the new way of programming look like what we are already used to.

What do you think? More or less separation to cope with scaling on multi-core processors?

Since quite some time StreamComputing has shared many research-papers on GPGPU, OpenCL, CUDA, WebCL and more via twitter. This activity has built up a priceless collection of PDFs, which actually got less easy to structure as the collection grew. Recently Evernote came to the rescue and slowly all these PDFs get archived in a way I can find back those papers concerning my current tasks.

As I noticed quite some people read (or at least favorite, share or download) the tweets/PDFs, I was always looking for ways to be able to share this growing library in a way it did not fall out of the short-term memory of Twitter. Evernote seems to be the best solution for this goal. The idea is that the notebook containing all OpenCL papers will be shared read-only with the subscribers. This means that you can search through related papers easily and have a short description of where it is about without putting time in archiving the PDF. Comments (by me and by people e-mailing me) will help you to determine if you should read it. Tags such as “paper”, “opinion”, “poster”, “thesis”, “fpga”, “cuda”, “webcl”, “memory”, give you easy ways to filter.

The idea is that in the near future this shared notebook (or a variation on it) will be a paid service (academic and other discounts available). To test its usability for this goal, I seek people who want to give feedback (read: fill in short questionnaires or send spontaneous mails with ideas). Interested? Just fill in the contact-form, mentioning “evernote”. Testers will get the service for free for at least 6 months. The results of the test I will share in this post.

I will continue posting all PDFs on twitter, so no pay-wall but a sort of freemium-model. The reason to ask money for the service, is that all hours spent on information-sharing has increased over the past year – choice is between reducing time spent on sharing, or offering services like this. If you think I should offer this service for free anyway or have other feedback already, please let me know in a comment or via the contact-form.

Edit: at stage 2, the PDFs will be shared via Dropbox or alike service. Problem is that it is writable for each member and more than 2GB, so only possible for paid accounts. Comment if you have ideas.

 With the launch of twitter-channel @OpenCLonARM we now officially show a strong interest in ARM for compute. And we are not the only ones, as the twitter already has 80 followers (60 in 1.5 day and 12 retweets of the welcome-message).

ARM has made tremendous progress in both technology and market-share. With ARM-64, companies like NVidia (and maybe AMD) in the field, X86 seems to be getting a real competitor. This could happen because since a few years computers are fast enough and are not being replaced by a faster one, but a smaller one (tablet, phone) or extra one. By the rules of the market, current technologies are replaced by the ones that give those other needs. ARM is fast (enough), flexible in design, very cheap, low-power and passively cooled. The biggest obstacle seems to be only getting a standard for a docking-station to connect your mobile, tablet or watch to keyboard, mouse and large screen.

OpenCL is perfect for ARM, as it gives the computation-power to the intensive computations not already covered by hardware-support. In the world of X86 this interests high performance and big data companies, where on ARM this interests also more. Without the need for OpenCL you can already watch HD video, with OpenCL you can encode the video with MP4. This year you will certainly hear more about new possibilities of OpenCL on ARM.

What do you think. Why does Intel not sell IP to ARM-companies as many technologies could be reused? Could Intel be the next ARM as an IP-seller, or will they stay the defender of X86 for many years to come?

StreamComputing.eu is not affiliated with ARM.

There is a lot going on at the path to GPGPU 2.0 – the libraries on top of OpenCL and/or CUDA. Among many solutions we see for example Microsoft with C++ AMP on top of DirectCompute, NVidia (and more) with OpenACC, and now AccelerEyes (most known for their Matlab-extension Jacket and libJacket) with ArrayFire.

I want you to show how easy programming GPUs can be when using such libraries – know that for using all features such as complex numbers, multi-GPU and linear algebra functions, you need to buy the full version. Prices start at $2500,- for a workstation/server with 2 GPUs.

It comes in two flavours: for OpenCL (C++) and for CUDA (C, C++, Fortran). The code for both is the same, so you can easily switch – though you still see references to cuda.h you can compile most examples from the CUDA-version using the OpenCL-version with little editing. Let’s look a little into what it can do.

Getting started

Note. If you use ArrayFire on Linux-64 with AMD, be sure you have at least AMD APP 2.5. Older drivers lock up your computer due to a bug in the AMD driver.

Be sure you have CUDA 4.0 or 4.1 installed for your NVidia GPU, and OpenCL 1.1 installed for your Intel and AMD devices. Check here to see how to get this going.

You can download ArrayFire from http://www.accelereyes.com/download_arrayfire after registering. You get both 64 and 32 bit libraries in one package and it installed without any problems. You can start right-away and go to bin32 or bin64, but if you want to recompile it then go the examples-directory and run “make clean && make”.

The compile-line gives an idea what it uses:

g++ -I../include -L../lib64 -Wl,-rpath=../lib64 -lafcl -LlibOpenCL.so.1 \
-LlibclAmdBlas.so.1 -LclAmdFft.Runtime.so.1.4.82 \
../examples/helloworld.cpp -o ../bin64/helloworld

You see it bundles the BLAS-library and FFT-library from AMD.

Let’s test if it installed correctly, by running “hello-world”. Results on my current PC:

Arrayfire (OpenCL alpha)

Device0: Barts (in use)
Device1: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Device2: GeForce GTX 560

The remark “(in use)” means the currently selected device for ArrayFire, not that it is in use by another process. If you don’t get any devices then you have not correctly installed the OpenCL drivers for your hardware.

Example: BlackScholes

BlackScholes is very popular demo-application for OpenCL and related techniques as it is slow as a single-threaded CPU version and very speedy when using OpenCL-capable devices. I wrote on it before in case you want to check how it looks like in OpenCL.

Benchmarks for Input Data Size = 184000 x 1:

• AMD: 1.939260 s
• Intel: 0.471669 s (around 0.1686 s in following runs)
• NVidia – OpenCL: TBD
• NVidia – CUDA: TBD

You can see the full code in the SDK, so I only focus on a few commands. All is in the namespace “af”, but I leave out “af::” before the commands. Under every command I describe what it does.

device(1);

This selects the second device (on this machine the Intel). Nice is that the rest of the initialisation is done when needed, so no real need to be very precise with it. Skipping this line selects device 0.

int N = 6;
float C1[] = {5.0f, 6.0f, 7.0f, 8.0f, 1.0f, 10.0f}; //different in code, 1000 times as long
array array1 = array(C1, N, 1);

This is all that is needed to produce an array of dimensions 6 x 1. If you would used N/2 and 2 as last two parameters, it would have given an array of 3 x 2 using the same data from C1.

for (int i = 0; i < iter; i++) {
   black_scholes(Cg, Pg, Sg, Xg, Rg, Vg, Tg);
}

This calls the “kernel” several times with arrays prepared as in the previous step. Below is the kernel.

void black_scholes(array& C, array& P, array& S, array& X, array& R, array& V, const array& T) {
   array d1_ = log(S / X);
   d1_ = d1_ + (R + (V * V) * 0.5) * T;
   array d1 = d1_ / (V * sqrt(T));
   array d2 = d1 - (V * sqrt(T));
   C = S * cnd(d1) - (X * exp((-R) * T) * cnd(d2));
   P = X * exp((-R) * T) * cnd(-d2) - (S * cnd(-d1));
}

Alternatively the code can just be called without separate function:

for (int i = 0; i < iter; i++) {
  array d1_ = log(Sg / Xg);
  d1_ = d1_ + (Rg + (Vg * Vg) * 0.5) * Tg;
  array d1 = d1_ / (Vg * sqrt(Tg));

  array d2 = d1 - (Vg * sqrt(Tg));

  Cg = Sg * cnd(d1) - (Xg * exp((-Rg) * Tg) * cnd(d2));
  Pg = Xg * exp((-Rg) * Tg) * cnd(-d2) - (Sg * cnd(-d1));
}

The nice thing of ArryFire is that arrays are overloaded such that computations are off-loaded to the selected compute-device. So “log(Sg / Xg)” is translated into optimised OpenCL-code and run on GPU or AVX/SSE. In this case each element of array Sg is devided by the element at the same location in array Xg. Log is called on each element of the resulting array.

af::sync();

Sync-function forces the compute-device to work.

You see. Once you get to understand the ArrayFire array class then you have all luggage needed to program an OpenCL-program.

Advantages & Disadvantages

I will get more into libraries built on top of OpenCL, to explain the common (dis)advantages. Any library has a scope for which it works best and products of AccelerEyes focus much on linear algebra and 2D data (arrays), less on 3D and images.

It is hard to say how much the library squeezes out the GPU(s) for you. But sometimes it works almost as good as manually optimised code and in many cases comparable, which is great as you get results much faster and can focus on your algorithm instead of coding. As a bonus each new version the optimisation gets better.

ArrayFire does not give you the generated kernels, so you depend on their license and libraries. Neither does it provide a fall-back to the CPU. This makes it a good solution for research and in-company deployments, but less for product development.

I am a big fan of splitting device-computations from the host-code. ArrayFire gives you the possibility to mix it again giving less optimisable code. But others will disagree with me.

Currently with version 0.3 it is still in alpha, so if you want to use it in production-code then you need to extensively test it. AccelerEyes has built up its good name in many years and is therefor careful before officially putting the library out of alpha/beta.

Learning more

I was happy to discover ArrayFire is so well documented and therefore you can accelerate your algorithm within a day. It depends on your exact situation and goals if ArrayFire is the right choice for you. The best is to give it a try and put your algorithm to the test. If you want to discuss the best option for your software, feel free to contact us.

StreamComputing has selected ArrayFire as one of the libraries on top of OpenCL with potential and therefore will start offering consulting-services for ArrayFire from April 2012.