I just found why my multithreaded python program was only being pegged to one core. Each thread needs to fight over the Global Interpretor Lock because only one thread can access the Interpetors objects/memory, which effectively makes your program singlethreaded.
Why the fuck would the developers put this in if it's not real!?
I knew I should of just wrote it in C.
The developers need to taken out back and beaten with a billy club, cause clearly their mamas didn't raise them right.
PROTIP: Stay the fuck away from this horrible steaming pile of shit they call Python.
If you disagree with me, you can go fuck your dad.
Python is a very good language when performance doesnt matter
Josiah Sullivan
No one asked you pajeet
Jose Powell
but bro, it JUST WERKS
Andrew Ortiz
true but multithreading is a meme anyway. use processes it just werks
Jason Green
I have a conspiracy theory that Python is promoted by HP, Dell, Supermicro and Huawei so shops have a reason to buy their 8 socket, 100TB RAM monstrosities for millions.
Benjamin Nguyen
This is a well known limitation of the python interpreter and it’s present in many other scripting languages (like lua) too. You can work around it with the multiprocessing library, but that’s only practical for jobs you can batch. The code executed in C libraries can also run in parallel, which is why Python is tolerable for data science. The GIL shouldn’t be an issue if you use Python for what it was intended to be: a scripting language. It only became a problem when people started trying to write servers and games in the language.
Adrian Green
i dont know people who use C
Kevin Gray
go back to plebbit
Sebastian Rivera
so? developer time is much more expensive
Tyler Thompson
you seem to not realize that the one on the far left is javascript.
James Barnes
Your fault for using Python to begin with I've already seen videos about this exact issue like 5+ years ago It's simply a joke/hipster language
Jordan Lewis
>
The threading module uses threads, the multiprocessing module uses processes. The difference is that threads run in the same memory space, while processes have separate memory. This makes it a bit harder to share objects between processes with multiprocessing. Since threads use the same memory, precautions have to be taken or two threads will write to the same memory at the same time. This is what the global interpreter lock is for. source: stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python
use right tool for right purpose, brainlet.
Gavin Wright
Pretty much all popular scripting languages have a GIL or an equivalent, though I admit they drive me up the fucking wall. Python threads are good for IO bound stuff and stuff that spends a lot of time in C modules. Other than that, check out the "multiprocessing" module for real concurrency with Python.
What's sad is, I liked perl and ruby a lot better than python, but python's just so easy that it won out, and now the other languages are dying a slow and painful death.
Nicholas Hernandez
all hail js all hail node the savior of software engineering
over the years people have invented many solutions to GVR queerness
in your case if the problem is math in nature just grab numpy with MT enabled backend, vectorize your algo and use numba
Asher Scott
Threads in most languages are assumed to be capable of executing in parallel in leave it up to the programmer to handle the complications. Multiprocessing is less efficient than threading because a process context switches requires remapping memory, while a thread context switch does not.
Adrian Powell
did you read the last line? >use right tool for right job if python doesn't work for your project, use language which does programming language is just a tool, hating on a language because it can't do what you want to do for your project is retarded
Grayson Roberts
or, u know, just use numba.
Ian Lewis
If you use python for anything else than scripting shit then you're doing it wrong.
But they always make me diamonds where else can i see some animations? Pretty much anything Studio Ghibli or Miyazaki is going to BD's thread to annoy him some more Unlabelled pics of bitches and pot.
Noah Perez
those all give the wrong answer what the fuck
Nathaniel Adams
This. Don't use python to write multithreaded programs that need to take advantage of multiple cores. Use something like Go or C.
Christopher Brown
KEK
William Jenkins
>purposely write code in such a way that the language does poorly for (You)s on a mongolian throat singing forum limit = 10_000
def prime(n): for i in range(2, n): if n % i == 0: return 0 return n
end = sum(map(prime, range(2, limit)))
print(f'sum of the first {limit} primes = {end}')
Joseph Harris
Coroutines provide concurrency, but not parallelism. Python provides a similar construct now with async. The work around in Lua is more efficient than Python, but it's still not quite the same as parallel multi threading.
Nicholas Morgan
guido is rolling in his grave
Dylan Price
He's dead?
Aaron Carter
Ya FP killed him. Now he's a zombie working on static typing for the language.
Gabriel James
You're right. This algorithm was used in a previous thread comparing js to c. I just copied the algorithm without modifying it to not change the test.
Charles Barnes
This was the thread.
Bentley Taylor
Also your algo is shit. You should only loop up to sqrt(n). Funny how to see Python haters not able to write 15 decent lines of code.
Other thing you should consider: how long to write your program? If it took you 40 seconds less in Python, then it's the winner. Your program is intended to be run once, so who cares if it takes more time to run?
Jace Murphy
Actually I'm not sure the algorithm is incorrect.
Liam Ortiz
I wrote none of the code. I just ran it to compare the times.
Also I can write code just as fast with javascript which runs much faster.
Jack Parker
:bigthink:
Nathan Kelly
dipshit
Logan Turner
What's the point of comparing programs badly written ? Python is not made for number crunching, that is well known. With your program, testing if 10000 is a prime number leads to 9900 useless comparisons since it is not useful to go over sqrt(10000).
Long story short, before trashing any language, learn how to design efficient algorithms.
Luis Lopez
>That guy who doesn't know how to make his python multithreaded
Gavin Williams
But the other language implementations do the same thing, changing the python implementation in a way that's better for python would defeat the purpose of checking how it plays out using that specific algorithm. So at least with that algorithm, python is slow.
Btw running that with pypy reduced it to ~4 seconds, much faster. Unfortunately pypy is not compatible with some libs.
Jeremiah Lee
Yes fellow user, a programming language created by a Netherlander is without a doubt a jewish conspiracy and being promoted by the big jewish owned corporations
What kind of retard uses Python for something CPU intensive?
Sebastian Richardson
I already write and prefer to write in C, but a coworker advised me that rewriting our shit python codebase into C was a waste of time and to multithread it to speed it up.
Cameron Bell
faggot who started this thread
Evan Rivera
All languages have their specificities. If you want a benchmark, then you need to compare programs correctly written in each language, not simply compare an algorithm.
Austin Gray
Here is an example of 3 algorithms to compute the sum of the 10000 prime numbers: pastebin.com/bs5QKdkb
One is the dump implementation, another is smarter by going up to n/2, and the third one is the smartest by going up to sqrt(n).
On my machine: is_prime_shit; Total: 496165411; Time spent: 0:00:18.667423 is_prime_half; Total: 496165411; Time spent: 0:00:09.712150 is_prime_sqrt; Total: 496165411; Time spent: 0:00:00.119127
0.1 seconds to perform this task efficiently. Even if C is still much faster (around milliseconds), does that still make sense to choose C over Python? If you need to run this task millions of times, yes. If you run it once in a while (like once every 10 seconds), then no.
Angel Hernandez
Python is designed for literal retards, don't use it if you want to do something other than abuse libraries. Lisp is better It's not so outlandish of a theory when you consider thaf Sun, HP, SGI and similar LOVED UNIX and X.org; no better excuse to sell more hardware than the software getting shittier and heavier every second
Michael Jones
do you realize it is a deliberately bad implementation to just test the running time of languages, right?
>not implementing a multithreaded sieve with multiple wheels and L2 cache optimization
Isaiah Hughes
>Why the fuck would the developers put this in if it's not real!? To facilitate intelkike shilling about "muh single core"
Lucas Wood
>lisp is better see, i always see this claim being thrown around yet not a single example of industrial strength lisp can be found.
Carter Collins
>should of SHOULD HAVE You actual retard
Hudson Wright
Lol Python. How the fuck are you going to get a job with that? Learn C++ you absolute morons.
Matthew Russell
Do you realize benchmarking shitty algorithms is not relevant, right?
Dominic Lewis
w-we could be friends if you wanted anonkun :3
Noah Carter
who are you? do you matter?
Camden Phillips
Ugh, don't be a dense fucking morons. Yes, benchmarking shitty algorithms *is* relevant because you are comparing the performance of a *particular* algorithm. The point is that, in real world applications, you are going to come across computations that are very similar to the shitty algorithms that are being benchmarked. Therefore you would want to compare using many different shitty algorithms. I'm repeating in order to get this through your thick skull because seemingly you have never taken a CS course or done much programming in an academic or professional setting.
Liam Richardson
Now that i think about it
>NodeJS is faster >NodeJS has almost the same libraries as Python >NodeJs has static typing with TypeScript >NodeJs is easy as fuck like Python
Yeah. Python is a language for the retards who are the reason why our modern supercomputers perform worse than the shitty machines of the 90s. >Who cares about performance, who cares about memory usage, all I care about is my developer experience!
Jaxon Kelly
they have had Julia for months now and haven't switched to it. Hell, people over here still use fucking Matlab of all things..
They are simply monkeys who just want the computer to spit out results of their models, and not waste time learning anything that goes out of their comfort zone they acquired in their undergraduate years.
Eli Rivera
>all I care about is my developer experience! typical MEMEMEMEMEME! millenial. they are used to mom & dad making everything easy for them
Charles Perez
Python isnt supposed to be used as your single language, i am pretty sure its actually decent when paired with C or C++ and easy to do.
Carson Morales
What the fuck is Julia
Lincoln Morales
basically a faster matlab.
Alexander Lee
I use assembler jump + move can everything
Landon Clark
No reason to switch to Julia if all of your libraries are still in Python.
Gavin Reed
I use R and it is miles ahead of any other language when it comes to statistical analysis. get fucked pyjeets and self proclaimed """""""""""""""""""""""""""""""""""""""""""data scientists""""""""""""""""""""""""""
Jeremiah Mitchell
>Being on a programming board >Using shit like R
Atleast Python is universal but R is just matshit for codlets, if you are here you could probably implement what you do in R in a normal language
Luis Turner
So your point is that you should start with an efficient language because people are going to write shitty algorithms anyway? Funny that this works in Python but not in C:
The second if forced killed after a few seconds, I guess you know why. However, the first one runs smoothly for hours. Does that mean C shittier than Python, then?
Or maybe your point is that it's better to change the language rather than improving the algorithm?
Then dude, I think you never step a foot in the professional world, because it's never gonna work this way.
Professionally, I worked with C, C++, Cobol, Java, Python and JavaScript in academic and private sector for more than 10 years and there is one common thing among all my jobs: the programming language was NEVER the problem.
Shitty algorithms (O(n2)) were the problem. Bad designs/architectures were the problem. I/O rates were the problem. Inefficient SQL queries were the problem.
Poor choice of language? No, never the problem.
Lucas Torres
>not knowing difference between multithreading and multiprocessing in puthon
Yikes
Owen Walker
This niggs thinks he can write his own implementation of MCMC
Benjamin Wright
>muh beloved puthon can do all shit of all trades, master of non.
>codelet might be, but if wanted to learn a programming language i would go for a real language like C or C++. and not waste my time with garbage that causes problems like in OP.
>tfw creating a Raspi Zero + e-ink screen project that needs to poll a CalDAV server once a minute for changes >tfw the two tools used to polling the server and creating the calendar output are written in python because there was no obvious alternative >tfw the Raspi Zero spends ten seconds every minute at 100% CPU capacity and runs at a toasty 45° at all times, meaning that I likely won't be able to wall-mount it without damaging the glue on my wallpaper
I fucking despise python. I tried using nuitka to compile this shit into a binary, but the Pi Zero actually doesn't have enough RAM to compile all of the modules into a standalone executable. And compiling just the original script does barely anything because python is a dependency shitshow where a 5kb script needs to draw in hundreds of megabytes of dependencies and interpret them dynamically to make a fucking web request. And the Zero uses a v6 ARM, while the more powerful Raspis that I own have v7 ARM, so I would need to actually cross-compile this shit to make it work.
Aiden Morgan
>using raspberry pi for anything else than putting it between your ass cheeks and posting pictures on the internet
Or try to understand the bottleneck since you might face the same with Node.js...
When something goes wrong, the first step is to understand WHY it goes wrong, not restart everything from scratch expecting it's gonna work better.
Tyler Lee
Translation: I'm too dumb to write anything and the only thing I can find is a retarded python module. And now this pajeet is mad at the language. The entire point of the rpi is that it's a shitty little computer for cheap. Of course it doesn't run everything, especially not poorly written python code.
Adam Flores
The thing is Node (and pretty much everything else) is much better optimized than Python so maybe it will work without completly burning the device cpu
Blake Russell
The Dutch are the biggest non-jew Jews in the world.
John Jenkins
How many Jow Forums users are actually Markov Chains or simple neural network outputs?
>which effectively makes your program singlethreaded. no, its threading is trash, but its not single-threaded effectively. you realise that multithreading exists for a single core, right? your threads simply timeshare
Dylan Butler
>should of Learn English first.
Landon Robinson
I'm not going to argue on which one is better optimized, but what user tries to achieve doesn't seem to require much resources. He is not doing CPU intensive computations, just some basic network requests. Any language should do the job on a Rpi zero.
Maybe user simply makes stupid requests, like trying to fetch the entire calendar at once: that would give the same shitty results on Node.js. Starting from this, the first step is to understand why it goes wrong. It's quite easy to spot where your program spends most of its time with tools like PyFlame or Py-Spy. Second step is to understand why these pieces of code are resource-hungry, and optimize them.
Wyatt Ortiz
I bet you didn't even use the multiprocessing module. Fucking amateur.
Ayden Parker
A Common Lisp implementation is both more concise and faster. You don't have to compromise.
Carson Collins
Multithreading in programming languages that don't have anything from functional programming is fucking trash anyway
> python shills lied, I cried It sucks it went that far, I use python but already knew about this. Take this as a lesson on why you can't trust freetards/lefties.
>slow language is slow The problem here isn't that python exists, but rather you miscalculating your project requirements. Here's a hint: if the device costs less than $50 it's probably not the fastest thing in the world. In other words it's your fault for not using a faster language (which is a fucking huge selection).
Angel Gonzalez
>wah I dont know what a green thread is and why it could be useful
Threading in a scripting language is hardly likely to be useful. Just spawn more interpreters and pass back messages.
Zachary Perez
Surprised you made it that far. I got to the part where I learned switch statements didn’t exist and alt+f4’d so fast. Meme language.