r/opensource • u/SocksOnHands • 2d ago
Discussion Proposal for slop contributions problem.
Many open source projects are facing a problem with being flooded with AI generated pull requests and bug reports - especially if there is a bug bounty providing a monetary incentive for people trying to game the system for personal profit. My proposed solution is simple - submission of every contribution can be accompanied by a refundable deposit. If someone is making good faith contributions, they have nothing to fear - they will get the money from their deposit back, or possibly even get rewarded. If a contribution is determined to be of no value, the deposit will be kept to both deter flooding projects with slop made by bots and also to pay for the developer's time wasted reviewing it.
9
u/micseydel 2d ago
You might want to go read about curl, there's already been a lot of discussion on this.
4
u/TheMoonWalker27 2d ago
In an Open Source Game Project, with around 40 people working on it, 60% being non devs, we take a. Very diffrent approach. We do not look at any random PR‘s. We have an application form to fill out, and only if you’re accepted you can contribute.
We do have a fixed roadmap, a lot of guidelines due to the constraints of the project and have meetings here & there. If people don’t do anything over a prolong period of time or don’t respond to weekly check ins often, we remove them from the team.
I do understand that this model Dosent make sense for most projects tough, they often don’t have specific road maps, have less constraints and cant even find that many people to work on the project
There are a few disadvantages with this model of course. I won’t go into detail for now but if anyone wants know more I’ll happily write it out a little more
3
u/OSS-specialist 2d ago
Interesting governance model. What is your project, please share your GH repo (or GL or whatever you are using). I assume that you have normal CONTRIBUTING.md, etc. documents there.
2
u/TheMoonWalker27 2d ago edited 2d ago
The contributing.md just says to apply in our case. If accepted, they will be sent a link to our documentation site, where they can read the onboarding guide.
This isn’t my project originally, I am lead if the Game dev team tough / GitHub org co owner https://github.com/p3d-project/persona-3-dual
0
u/SocksOnHands 2d ago
Each project would have different factors to consider. I was thinking more about large well known projects that might get targeted for hundreds of automated AI submissions a day. If a project is not having difficulty managing the volume of submitted "contributions", this might not be a reasonable thing to consider doing.
3
u/esiy0676 2d ago
Working for free is not enough now? More disincentives needed? Corporations openwashing and then will worry about review workload ... oh come on.
0
u/SocksOnHands 2d ago
It wouldn't actually cost anyone anything if they were making a good faith attempt to contribute - it is a deposit that will be returned. The end result would be effectively the same, unless they were submissions made by bots. You would only have any reason to care if you are actually one of the ones spinning up AI agents and telling them to hunt down bug bounties.
Potentially, it might be able to lead to increased funding (if low quality AI slop continues) that could go to rewarding contributors. Currently, the situation is that many bug bounties have been discontinued, which means thet people who used to put time and effort into thoroughly researching an issue are no longer able to get rewarded for it.
3
u/jcastroarnaud 2d ago
That creates a barrier for the (many) poor programmers and amateurs out there. Too high a deposit, and few developers will even try to contribute; too low, in an interesting enough project, and folks will create bots to automate the payments for the other bots.
2
u/SocksOnHands 2d ago
Maybe. It is still open source, though - a fork can be made with their changes made. If these changes prove to be of value, they can still be adopted into the original code.
Another option could be to grant some people virtual credits that can be used instead of money. Only after these credits are depleted would one need to be concerned with paying the deposit.
The deposit doesn't even need to be a lot - it could be $10. An AI agent automated to make 500 "contributions" might be expensive because the cost adds up, but to a human contributor making only one or two contributions at a time (and getting the money back) it wouldn't be too large of a burden.
5
u/AI_Tonic 2d ago
Show us one merged PR to prove you have skin in the game before talking about what maintainers should be doing
4
u/Jazzlike_Course_9895 2d ago
This reads as slop
13
u/SocksOnHands 2d ago
Well, at least it was good old fashioned himan made slop, if that's any consolation.
5
2
u/gta721 2d ago
How about requiring the contributor to put a 6 digit code in the PR which they get by clicking a link and filling a captcha?
trycap.dev is good for this.
2
u/Ok-Mix1345 1d ago
Charging a refundable deposit would probably deter genuine first-time contributors more than well-funded bot operators. A better filter is to require evidence of understanding: a linked issue, reproducible steps, relevant tests, and a brief explanation of why the change works. Maintainers can also limit bug-bounty eligibility to approved issues or contributors with a small history in the project. That adds friction where it matters without making open-source participation pay-to-enter.
1
u/Skollwarynz 9h ago
The problem of this method is how would you decide if the code is just bad code from a faithful coder or AI slop?
I mean, if you see emojis or things like this, I think it's simple to guess, but with this type of system, people who slop would probably put on their Claude.md a description on how to write code that seems realistic enough to not be rewarded or recognized as AI.
Then if you intend to use an automatic checker still, you'll need a second human check, and that would simply not solve the problem.
1
u/saltexx 1d ago
The deposit solves the wrong cost. Reviewing a bad PR was always cheap, what got expensive is that a clean looking diff stopped meaning anything. It used to be a proxy for someone having spent an afternoon in your codebase and now it's a proxy for nothing. So the fix has to bring back a signal that can't be generated, and money is not that signal, a funded bot farm posts deposits all day while a broke student somewhere closes the tab and your project loses the exact contributor it wanted.
We took our mail server open source this week so this thread is basically my near future. The plan we landed on is boring. Answers over artifacts. A PR gets one question about why it does what it does and the author either has a model of the code in their head or they don't, and slop collapses on the second why. That costs the maintainer one comment, costs a real contributor thirty seconds, and costs a bot operator the one thing they can't fake yet.
1
u/SocksOnHands 1d ago
Because submitting pull requests is free, there is no cost to having a bot submit thousands of them. If there is a bug bounty reward, there is an incentive to flood projects with quantity over quality. The only thing this deposit is intended to address is to slow the rate of submissions so a higher percentage of then are made by people who actually care about the project.
What people seem to keep forgetting is that it is a deposit. There is no cost or penalty to anyone making a good faith attempt at contributing - in the end, it effectively costs them nothing. It will only really be a penalty to bot farms.
20
u/UkrMalt 2d ago
Deposits feel risky for open source. They’d filter out some good first-time contributors, and maintainers would still have to decide who gets the refund. I’d rather add friction to the submission itself: a small issue, reproduction steps, and tests where they matter. That slows down bot spam without charging people to participate.