0

I have acquired some extra raspberry pi's and am looking to make use of them all in some way.

As I have been reading around, all of the "turn your Pi into a supercomputer" articles are only referring to using software like MPI to split up tasks and send them to each of the processing nodes.

However, I cant seem to find a lot of information on if it is possible to set up two or more raspberry pi's such that they all contribute their processing power, RAM, Storage .etc to some sort of shared pool that then can be used as if it were a single raspberry pi with better specs.

I seem to remember something similar to this being possible with docker containers (you could run containers in an environment made up from a shared pool of a whole cluster of resources). Is there I could setup on my pi's that would allow me to set them all up in a cluster to create one beefier "virtual pi" that I could use as if it were just a single normal pi?

DeveloperACE
  • 101
  • 2

1 Answers1

1

Think about this question "Is it possible to connect my laptop to my partner's laptop to double the speed of running that steam game I've just downloaded?".

The answer is the same.

To exploit clustering you need to divide the workload into pieces, schedule each piece to run on a member of the cluster. Then when each is done aggregate the results into a single coherent results set.

That needs specialised programming to exploit your cluster.

Dougie
  • 5,301
  • 10
  • 19
  • 28
  • to use your analogy, What about an eGPU? you can plug an external GPU into most laptops now to increase their performance in games. Would it be possible to have one raspberry pi as the master thats connected to the other nodes such that the other nodes are just resource passthroughs essentially? – DeveloperACE Jan 14 '19 at 23:32
  • i'm trying to essentialy make a server that can run basically everything that a regular linux machine (rpi included) can. the type of cluster you describe seems like its application specific. i want to be able to do anything with this server that i could with a regular pi (discord bots, minecrat server, web hosting, Pihole, PFSense whatever) – DeveloperACE Jan 14 '19 at 23:35
  • 1
    You're missing the point. The cluster needs a specialised workload or you gain nothing with a pair of loosely coupled Linux machines that have a 480Mb/s I/O limit. – Dougie Jan 15 '19 at 00:28
  • Yeah, that’s what I was reading in other posts on this site. I was just wondering if it was possible to do this sort of thing with a pi cluster since I have some unused pi’s and I vaguely remember other pieces of software with similar functionality. – DeveloperACE Jan 15 '19 at 03:10
  • The main goals I am trying to achieve are a) to be able to run various workloads like a server (discord bots, web servers whatever) and b) to only have one “interface” to the server (I.e. I do t want to have to change the same thing on every single pi every time. I just want to manage it as if it were one pi but be able to not have the rest of my pi’s just sitting there doing nothing – DeveloperACE Jan 15 '19 at 03:13
  • I understand a cluster probably isn’t the way to do this so I’m thinking something more like load balancing but I can’t seem to find anything that would work particularly well for this use case – DeveloperACE Jan 15 '19 at 03:15
  • Load balancing is just a form of specialised workload. You split into chunks and balance the chunks across the servers. That needs a high speed shared disk subsystem (which you don't get on a RPi) or you have to break the workload into chunks based on where the resources are available. – Dougie Jan 15 '19 at 18:43
  • Would it be possible to use GPIO as a faster bus for data transmission than standard networking? – DeveloperACE Jan 16 '19 at 04:41
  • 1
    Not possible. The GPIOs are not a bus. They carry slow things like SPI & I²C. Fastest buses are CSI & DSI but they are dedicated to the camera & display. – Dougie Jan 16 '19 at 09:39
  • And I presume it isn’t possible to use those for this purpose either? – DeveloperACE Jan 16 '19 at 15:41