Some more here:

https://storage.courtlistener.com/recap/gov.uscourts.cand.40...

https://storage.courtlistener.com/recap/gov.uscourts.cand.40...

https://storage.courtlistener.com/recap/gov.uscourts.cand.40...

Friendly people.

I've received emails like that too over the years. What hugely controversial thing do I do? I have a website where I sometimes write about $stuff and I post on HN. Keeping the basic info private is probably a good thing especially if they're based in the US, because "SWATting" etc, but beyond that it doesn't seem "credible" in the sense that it's very likely someone will show up at their door with a gun.

Since the first two are redacted, I wonder if they sent them with their real names.

An impressive number of 4chan's /g/ users are on github. Some even actively contribute to Linux (though usually to Arch, Gentoo, and now more and more Nix).

I wrote a paper during college that I should release some time about when /g/ threw an absolute shitfit over Linus going "so, I've been a kinda shit human being to people and I'm going to step back and get some help", going as far as to blame his daughter/"the woke mob"/multiple named core kernel contributors for killing their god.

At one point, I attended a GitHub event that wasn't directly sponsored by github but encouraged a lot of github users to show up. While there I met several people who, outside the venue, were talking animatedly about Terry Davis. Listening in on the conversation revealed that they more or less just approved of his extensive use of racist language and epithets.

I haven't checked, but I would suspect that Linus' recent "trans rights" by proxy post has caused at least one or two aneurisms in the /g/ user group.

Considering how large GitHub is (in the industry) it’s like asking is “are Facebook users gamers”?

No, but I assume a lot of AI bros are

> What would you say the basis of all knowledge you know is? You are a collection of everything you have consumed and the stuff you create is all influenced by that.

FWIW, humans certainly can infringe other peoples' copyrights and can do so even if they aren't actively intending to do so. There is some boundary across which you are no longer just learning something and you are now copying, and it isn't clear at all that these generative AI techniques are actively considering the latter the way a human is required to.

But, sure: if you are against the idea of copyright entirely then it is hard to consider the idea inconsistent, though I would think a world without copyright would be a particularly hard one for an artist to make money at all...

>What would you say the basis of all knowledge you know is? You are a collection of everything you have consumed and the stuff you create is all influenced by that.

Surely you're not suggesting that there's no such thing as "original work". The production of which may have very high capital and labour costs - which if not protected from theft - would remove the incentives of producing original work.

>As someone who very much has skin in the game(my art is trained on midjourney)

I don't know your specific situation, but there's obviously different scales of importance here. What if your art was your sole source of income, and people were reproducing it under their own name? or if you had a product where you poured millions into developing some novel IP/methods, and some employee brought it with them when they went to work at your competitors?

some things like drug discovery could probably be done with a bounty system rather than intellectual property, and could probably get much better results for a fraction of the cost for maintaining the intellectual property component of the court system

The ones who are excited about these as tools are the ones who are openly inspired and want to inspire however the ones who claim copyright infringement seem to come off as insecure

Yeah yeah, your side are the good guys and the other side is a bit dodgy.

> What would you say the basis of all knowledge you know is?

When someone teaches me, they don't own all my future creative output.

When someone teaches an AI, they do.

That's the principal difference between human learning and machine learning.

I think what you suggested is an unpopular opinion, but I also wholeheartedly agree with it. :)

I'm certainly no expert on copyright law, but my understanding is that its purpose is to protect the financial interests of certain creators from the progress of technology (e.g. copy paste). I've heard arguments that removing copyright would lead to less creativity or reduced quantity or quality of work, but I'm personally a bit skeptical (probably for the same reasons as you - I think people have a natural desire to create). Even in terms of financials, I would speculate that an employment/patronage model would become more widespread.

I think there's something to be said about the benefits of having freely available knowledge, music, and art for common consumption. When I was a child in high school (or well, always lol), my parents couldn't afford a lot of material I needed or wanted for studying (especially for standardized testing, SAT and AP tests) and most of the books in my local library either did not exist or were outdated. But when I discovered that much of this information could be found online, it really changed my world and made success in life feel attainable to me. I consider myself quite wealthy now, but I don't think I would have been able to escape poverty if all this information was paywalled from me. Maybe others would argue the writers are not being compensated for their efforts, but if there are other people in the world in the same position as past me who could positively benefit from it, I think that's a better world to live in, personally.

Incidentally, the release of StableDiffusion has actually inspired me to draw a little. Not sure why, but I find it inspiring being able to iterate on a prompt and produce something of quality that I can try to replicate on my own. Even if I fail, I still have something to appreciate that maps fairly well to the concept in my head.

My hope is that these technologies might lead to a change in our financial system (I think UBI would be a good idea), but I suppose we'll see where everything ends up. I think there's likely going to be a lot of pain in the short-term (especially since there are those who don't want to adapt), but hopefully everyone will positively benefit in the long-term.

Model weights are a form of data compression. Have a look at variational autoencoders for example where this is made more explicit (its latent space is a compression).

Your arguments regarding "transformative work" is what the discussion is all about. Let's see where it lands with case law.

FWIW, my stance is copyrighted content should not be used in training without request.

>It's bad practice to go around espousing strong and condemnatory opinions about topics you don't have a full grasp of. In this case, it's both the technical details and the legal system. >It makes you look like a fool and costs you your credibility amongst peers in future encounters.

Agree, thank you for the feedback. My analogy was quite exaggerative.

> It's bad practice to go around espousing strong and condemnatory opinions about topics you don't have a full grasp of.

I disagree, the internet wouldn't be half as full of knowledge as it is if it weren't for the loudly ignorant giving the experts somebody to correct.

Afaik the currently best compression techniques use a neural network internally. So the distinction is perhaps not as clear as you might think.

The lowest temperature isn't always the one that results in working code! This was shown in the original Codex paper:

> When evaluating pass@k, it is important to optimize sampling temperature for the particular value of k. In Figure 5, we plot pass@k against the number of samples k and the sampling temperature. We find that higher temperatures are optimal for larger k, because the resulting set of samples has higher diversity, and the metric rewards only whether the model generates any correct solution.

> In particular, for a 679M parameter model, the optimal temperature for pass@1 is T∗ = 0.2 and the optimal temperature for pass@100 is T∗ = 0.8. With these temperatures, we find that pass@1 and pass@100 scale smoothly as a function of model size (Figure 6).

So even with pass@1 (likelihood of getting the right answer in 1 attempt) you don't use T=0, so there will be slight variations in the output each time.

Well, temperature 0 means the completion is always the most "likely" (or "best", after fine-tuning) token, while temperature 1 means to choose the next tokens stochastically according to their probability (or "goodness" after fine-tuning). Usually some temperature in between is chosen, like 0.7. It's not a priori clear to me which is the best way to do it.

Generally the reason behind adding randomness to machine learning is avoiding "local minima" in the search space of the optimization function(s) used for training the model. If your training produces a very smooth descent to an optimum it can lead to the model converging on a solution that is not globally the best. Adding some randomness helps to avoid this.

Specifically for GPT models, the temperature parameter is used to get outputs wihch are a bit more "creative" and less deterministic. https://help.promptitude.io/en/ai-providers/gpt-temperature

I don't know much about AI, but I think one reason you might do that is to learn which variations are preferred (which are committed unmodified) so you can tune the model in the future. I don't know if Github does that, but given they've cited how often code from copilot is committed without modification, I assume they are measuring it at least in some cases.

Huge topic, worth Googling. Short version is that too little randomness limits the solution space, so retrying suboptimal results yields the same problems.

Potentially more correct, yes. It frees the model to choose lower probability tokens to some degree, technically it boosts their probabilities, which may be more correct depending on the task.

There are also sampling schemes, top_p and top_k which can each individually help choose tokens that are less probable (but still highly probable) but more correct, and they are often used together for the best effect.

And then there are various decoding methods like beam search where choosing the most optimal beam may not mean the most optimal individual token.

By default a simple greedy search is used which always chooses the next highest probability token.

It is worthwhile with creative writing. For example if you ask ChatGPT to write a short story, you want some originality. Even when asking for an explanation it can be useful as you may want to try different things for the explanation that speaks to you the most.

But here we are talking about autocompleting code. I don't think programmers want the autocompleter to be creative. They want the exact same solution everyone uses, hopefully the right one, with only minor changes so that it matches their style and use their own variable names. In my case, I am the programmer, I decide what to do, I just want my autocompleter to save me some keystrokes and copy-pasting boilerplate from the web, the more it looks like existing code the better. I have enough work fixing my own bugs, thank you.

Speaking about bugs, how come everyone talks about code generation that, I think, doesn't bring that much value. Sure, it saves a few keystrokes and copy-pasting from StackOverflow, but I don't feel like it is the thing programmers spend most of the time doing. Dealing with bugs is. By bugs, there are the big ones that have tickets and can take days to analyze and fix, but also the ones that are just a normal part of writing code, like simple typos that result in compiler errors. I think that machine learning could be of great help here.

Just a system that tells me "hey, look here, this is not what I expected to see" would be of great help. Unexpected doesn't mean there is a bug, but it is something worth paying attention to. I know it has been done, but few people seem to talk about it. Or maybe a classifier trained on bug fix commits. If a piece of code looks like code that has been changed in a bug fix commit, there is a good chance it is also a bug. Have it integrated to the IDE, highlight the suspicious part as I type, just as modern IDEs highlight compilation errors in real time.

Ye olde Bias-Variance tradeoff

Yes.

Why would you ever write imports? IDEs autocomplete them for you

Copilot really messes up imports for me sometimes. The ide does a better job via autocomplete when it is turned off

> this isn't some example from a textbook or even stack overflow, but licensed pieces of work with all the legal complications that come with that

I understand why these might feel different to you, but textbooks and stack overflow are also proprietary, licensed pieces of work. I don’t see why there would be much of a legal distinction.

No, you're missing the point.

There are two worlds.

In one, everytime someone publishes code with a license attached, they've taken a chunk out of the set of valid lines of software capable of being permissibly written without license encumberance. This is the world the poster you are replying to is imagining we're headed toward, and this case basically does a fantastic job of laying a test case/precedent for.

The other world, is one where everyone accepts all programming code is math, and copyrighting things is like erecting artificial barriers to facilitate information asymmetry. I.e. trying to own 2 + 2. In this second hypothetical world, we summarily reject IP as a thing.

The 2nd world is what I'd rather live in, as the first truly feels more and more like hell to me. However, given the first one is the world we're in, I'd like to see the mental gymnastics employed to undermine Microsoft's original software philosophy.

EDIT: Voir dire will be a hoot. Any wagers on how many software people make it onto the jury if any?

what would happen without that hyperlink? the overall issue seems to be a lack of attribution to the originator.

How do you like to name varibles. Do you use constant == variable or variable == constant. Tabs vs spaces. Declarations inline with first use, or at the top as K&R intended. Comment syntax and content. Etc.

Lots of little things.

Depending on language there are ton of style choices. There’s style guides as examples of trivial.

Non trivial include names, comments, logging, error checking, structure, ordering of operations that aren’t sequential.

I interpreted the comment you are responding to as “make sure it uses the same style conventions as the rest of this file”, which is something that Copilot does very well!

this comment really hits hard for me -- its like there is a place to buy food where every menu item is clearly shown, with a large color picture and a printed price.. and the person talking has only every purchased food in that way.. as if there are no alternatives that "really exist"

there really are a lot of other scenarios that involve writing software, to make software. Its not possible to list them all.. the list changes while I type

Nobody affirmatively says “yes, my goal is to violate copyright law, and Copilot is the best tool I’ve found”. But it doesn’t seem impossible to me that the value of Copilot comes partially from the fact that it can copy paste code from copyrighted repositories in ways which would be illegal for you or I to do. I’m not sure it’s proven yet but I wouldn’t be shocked if it is in the future.

Copilot is a product -- at least indirectly -- of Microsoft, a company who for about a decade made very public pronouncements about how they disagreed with the GPL (or copyleft generally), found it problematic, and tried actively to discourage its use.

Today's MS isn't really the same, and they've clearly made their peace with Linux. But it still happens that the GPL is in some fundamental ways at odds with commercial exploitation of open source code. So any corporate entity is going to struggle with it because at best it requires being very careful in distribution, or trying to negotiate or cut a deal with the licensee. At worst it can lead to legal problems and IP leakage on your own product.

So, not claiming any conspiracy. Or intent to violate intentionally. But it is in the convenient interests of companies like MS/OpenAI/GitHub to treat open source work as effectively public domain rather than under copyright, and to push the limits there.

The risk to an employer is of course the accidental introduction of such copylefted material into their code-base through copilot or similar tools.

I suspect two sources of disconnect with the broader community on hackernews that doesn't seem to see the issue here:

a) Much of the folks on this forum are working in the full-stack/web space where fundamentally novel, patented, or conceptually difficult algorithms and datastructures are rare. For them Copilot is an absolute blessing in helping to reduce the tedium of boilerplate. However in the embedded systems, operating systems, compiler, game engine dev, database internals etc. world there are other aspects at work. In certain contexts, Copilot has been shown to reproduce complicated or difficult code taken from copyrighted or copylefted (or maybe even patented sources) without attribution. And apparently now with some explicit obfuscation.

To put it another way: it's unlikely that Copilot's going to violate licenses with its assistance with turning your value/model objects from one structure to another, or writing a call into a SQL ORM. But it's quite possible that if I'm writing a DB join algorithm or some complicated math in a rendering engine or a compiler optimization phase that it could "crimp notes" from a source under restrictive license... because those things are absolutely in its learning set and the LLM doesn't "know" about the licensing behind them.

b) Either misunderstanding of, or lack of knowledge of, or outright hostility to... copylefted or attribution licenses which require special handling.

Sorry your employer forbids the use of tooling that makes your life better and reduces drudgery. Perhaps you should vote with your feet and find a less Luddite employer.

Does your company allow you to outsource your work to people in a poorer nation for a fraction of the cost that you are paid? Why not? Perhaps you should vote with your feet and find a less Luddite employer.

My company forbids the use of LLMs that aren't validated (and we make one).

Our managers get emails if we make calls to known LLMs, and there's guidance on locally running LLMs and using their output ("it's okay for small things maybe, but be careful"). Why?

Because legal's job is to protect the company from legal threats. Sometimes that means making some awkward choices, like handwringing over the use of GPL licensed software in publicly exposed example code (such as sample apps) purely because some aspects of the GPL haven't been tested in American courts, much less international ones.

So the use cases for LLMs there are mostly source-to-source transformative ("Turn this function and documentation into javadoc format please") or similar -- stuff where you can show that the LLM isn't introducing anything that might maybe possibly have any hint of externally licensed software.

[flagged]