The main issue I see is that papers are actually becoming so focussed on form that they are now unreadable. People prefer reading my blog for my papers than reading the papers themselves. In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
The other side of all this academic brownie points via papers (and doing reviews, which has become "brownie points for gatekeeping") is that most academic software is not only unmaintained, but actually unusable. They rarely even compile, and if they do, there is no --help, no good defaults, no README, and no way to maintain them. They are single-use software and their singular use is to write the paper. Any other use-case is almost frowned upon.
One of the worst parts of Academic software is that if you re-write it in a ways that's actually usable and extensible, you can't publish that -- it's not new ("research") work. And you will not only have to cite the person who wrote the first useless version forever, but they will claim they have done it if your tool actually takes off.
BTW, there are academics who don't follow this trend. I am glad that in my field (SAT), some of the best, e.g. Armin Biere and Randal Bryant are not like this at all. Their software is insanely nice and they fix bugs many-many years after release. Notice that they are also incredibly good engineers.
Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors. CRYPTO, the annual "S-tier" conference in California, has already let people attend virtually online, and is considering its options for next year.
Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
The whole show up to conferences internationally to network and put attendace on your CV thing is also not great for people looking after children, among others.
In practice if you want discussion and citation for your cryptography paper, it has to go on IACR eprint at some point. Being published in CRYPTO is still a major endorsement, but not the way people actually get hold of a copy these days.
> Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors.
It seems to get a lot more attention now that people from a different type of country are getting affected.
> Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
Do you mean safe for individuals or a choice of venue?
In the UK (which is the country I know best) individuals are fine once they get a visa, but its not a safe choice in terms of planning because the granting of visas for people from certain countries is unpredictable (so people you expect to be able to attend might not be allowed to).
Note how the author talks about "main conferences", "major conferences", and "top conferences". That's the root issue. Whenever there is prestige available, people will compete for it. And if you have a competition, you should formalize the rules to make it fair.
When I was doing PhD ~15 years ago, I noticed that I rarely cited work that appeared in the top conferences of the subfield. Those conferences covered so wide range of topics that often only 1 or 2 papers were in the same subsubfield as me. And even those were often not directly relevant to my work.
But then there were small specialized conferences that had plenty of interesting papers every year. I left CS for another field a decade ago, but I still regularly attend some of those conferences and review for them. The papers published in them are still interesting and relevant to my work.
A big part of the problem here is that Universities have increasingly begun attaching prestige to specific “top” conference publications for both ranking and faculty promotions. A good example of the phenomenon can be seen in [1] (sorry for the noun-citation!) which only gives credit for approximately three conferences in each field. Combine this with a flood of new researchers entering CS, you have a recipe for “top” conferences being essentially destroyed and filled with uninspired work.
(And contrary to the joke in the article, even your own work becomes uninspired when you ship it to those conferences. You can’t afford to be quirky or interesting.)
Fortunately every field has a fourth or fifth-tier conference that isn’t on this list (or a specialized topic conference that the rankings folks don’t care about), and those still serve the purposes that conferences were made for. You just might not be able to convince a ranking-obsessed administrator that your work has any value if you publish there.
I think the example of how to "correctly" cite a paper actually makes this issue seem smaller than it is. In reality, these conferences have very complicated (and unstated) "rules" for how a paper is supposed to look. If an "outsider" wanders in and submits a paper with new ideas, it will be very obvious that they are not a "member of the community" and their paper will usually be treated much more harshly as a result. This adds a huge amount of friction to research.
And what's particularly frustrating is that many organizers will try to combat this by writing papers saying they "particularly encourage" papers that are interdisciplinary, or focused on less fashionable topics, etc. It's good that they are trying to change things, but I think the main effect in practice is to encourage people to spend their time writing papers that have little chance of being accepted.
This issue isn't at all unique to computer science, though. Try publishing a paper in a top economics journal as an outsider!
I am fairly certain this rule was there against an obnoxious citing style of "The lambda calculus [1] was intended as a foundation for mathematics". It is especially obnoxious in the case of CS because when you cite e.g. "as Johns comments in his article about future developments of the programming languages [1963a]" it is quite important to know that this paper is actually from 1963 and can be mostly disregarded except as a historic curiosity; yet I've seen people vehemently defending this "[1]" style.
The only time I like numbers is writing proposals and I only like it because it saves space. Other than that I much prefer (name, year) if I am to have a preference at all.
Adding to the frustration, (the lack of) these shibboleths partially undermine double-blind reviewing, which is on the rise in prestigious conferences. A reviewer from the in-group may immediately spot that a submission comes from the out-group.
Yes papers not having working code is very annoying and anti intellectual. I wrote a substack about this recently because I review a lot of papers that essentially are trash because the authors do not provide working codes:
In AI/ML job ads it is quite typical for the requirements to include, "must have published at top AI conferences include but not limited to NeuroIPS, ICML, ICLR, ...etc", which I find completely crazy, because it just incentivizes grad students to publish rubbish papers to have on their CV, and indeed most conference papers in AI/ML are complete rubbish, because it is trivial to take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper. Even dud results are published as "promising". It's just a complete shitshow, and as somebody who is in the field it feels as though you cannot even complain because people get offended. You're just supposed to keep spinning the hamster wheel without posing any hard questions.
Moreover, having published at top conferences does not prove that somebody is going to be a good ML Engineer. It just proves that somebody knows how to write a compelling conference paper, which is a completely different skill set altogether.
The main issue I see is that papers are actually becoming so focussed on form that they are now unreadable. People prefer reading my blog for my papers than reading the papers themselves. In fact I hear people telling me they understood the blog _better_. The whole academic writing shtick has become so obtuse that not only writing is cumbersome, but so is reading.
The other side of all this academic brownie points via papers (and doing reviews, which has become "brownie points for gatekeeping") is that most academic software is not only unmaintained, but actually unusable. They rarely even compile, and if they do, there is no --help, no good defaults, no README, and no way to maintain them. They are single-use software and their singular use is to write the paper. Any other use-case is almost frowned upon.
One of the worst parts of Academic software is that if you re-write it in a ways that's actually usable and extensible, you can't publish that -- it's not new ("research") work. And you will not only have to cite the person who wrote the first useless version forever, but they will claim they have done it if your tool actually takes off.
BTW, there are academics who don't follow this trend. I am glad that in my field (SAT), some of the best, e.g. Armin Biere and Randal Bryant are not like this at all. Their software is insanely nice and they fix bugs many-many years after release. Notice that they are also incredibly good engineers.
Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors. CRYPTO, the annual "S-tier" conference in California, has already let people attend virtually online, and is considering its options for next year.
Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
The whole show up to conferences internationally to network and put attendace on your CV thing is also not great for people looking after children, among others.
In practice if you want discussion and citation for your cryptography paper, it has to go on IACR eprint at some point. Being published in CRYPTO is still a major endorsement, but not the way people actually get hold of a copy these days.
> Unfortunately it's becoming an increasing problem that travel is not equally safe depending on your nationality, destination, and other factors.
It seems to get a lot more attention now that people from a different type of country are getting affected.
> Any conference that announces itself as being proudly diverse and inclusive will have to have some difficult board meetings this year. It's not just the US, there's several countries in Europe that need a closer look at too. I hear Canada and the Nordic countries are fairly safe.
Do you mean safe for individuals or a choice of venue?
In the UK (which is the country I know best) individuals are fine once they get a visa, but its not a safe choice in terms of planning because the granting of visas for people from certain countries is unpredictable (so people you expect to be able to attend might not be allowed to).
Safe for individuals, mainly.
The UK right now is also trying to figure out who can use what bathrooms. I don't understand the details myself.
Note how the author talks about "main conferences", "major conferences", and "top conferences". That's the root issue. Whenever there is prestige available, people will compete for it. And if you have a competition, you should formalize the rules to make it fair.
When I was doing PhD ~15 years ago, I noticed that I rarely cited work that appeared in the top conferences of the subfield. Those conferences covered so wide range of topics that often only 1 or 2 papers were in the same subsubfield as me. And even those were often not directly relevant to my work.
But then there were small specialized conferences that had plenty of interesting papers every year. I left CS for another field a decade ago, but I still regularly attend some of those conferences and review for them. The papers published in them are still interesting and relevant to my work.
What are you doing now?
A big part of the problem here is that Universities have increasingly begun attaching prestige to specific “top” conference publications for both ranking and faculty promotions. A good example of the phenomenon can be seen in [1] (sorry for the noun-citation!) which only gives credit for approximately three conferences in each field. Combine this with a flood of new researchers entering CS, you have a recipe for “top” conferences being essentially destroyed and filled with uninspired work.
(And contrary to the joke in the article, even your own work becomes uninspired when you ship it to those conferences. You can’t afford to be quirky or interesting.)
Fortunately every field has a fourth or fifth-tier conference that isn’t on this list (or a specialized topic conference that the rankings folks don’t care about), and those still serve the purposes that conferences were made for. You just might not be able to convince a ranking-obsessed administrator that your work has any value if you publish there.
[1] https://csrankings.org/
I think the example of how to "correctly" cite a paper actually makes this issue seem smaller than it is. In reality, these conferences have very complicated (and unstated) "rules" for how a paper is supposed to look. If an "outsider" wanders in and submits a paper with new ideas, it will be very obvious that they are not a "member of the community" and their paper will usually be treated much more harshly as a result. This adds a huge amount of friction to research.
And what's particularly frustrating is that many organizers will try to combat this by writing papers saying they "particularly encourage" papers that are interdisciplinary, or focused on less fashionable topics, etc. It's good that they are trying to change things, but I think the main effect in practice is to encourage people to spend their time writing papers that have little chance of being accepted.
This issue isn't at all unique to computer science, though. Try publishing a paper in a top economics journal as an outsider!
I am fairly certain this rule was there against an obnoxious citing style of "The lambda calculus [1] was intended as a foundation for mathematics". It is especially obnoxious in the case of CS because when you cite e.g. "as Johns comments in his article about future developments of the programming languages [1963a]" it is quite important to know that this paper is actually from 1963 and can be mostly disregarded except as a historic curiosity; yet I've seen people vehemently defending this "[1]" style.
The only time I like numbers is writing proposals and I only like it because it saves space. Other than that I much prefer (name, year) if I am to have a preference at all.
Adding to the frustration, (the lack of) these shibboleths partially undermine double-blind reviewing, which is on the rise in prestigious conferences. A reviewer from the in-group may immediately spot that a submission comes from the out-group.
The publication of scientific papers is broken too:
1. authors that just reviewed the paper, did not do anything substential 2. papers that do not ship with working code 3. papers that are meaningless
Yes papers not having working code is very annoying and anti intellectual. I wrote a substack about this recently because I review a lot of papers that essentially are trash because the authors do not provide working codes:
https://open.substack.com/pub/mnky9800n/p/how-to-format-code...
A great link nugget from the article: https://wp.doc.ic.ac.uk/cairesfe/wp-content/uploads/sites/80... - a parody on how some historically very significant papers might be rejected in today's system. made me chuckle a lot
In AI/ML job ads it is quite typical for the requirements to include, "must have published at top AI conferences include but not limited to NeuroIPS, ICML, ICLR, ...etc", which I find completely crazy, because it just incentivizes grad students to publish rubbish papers to have on their CV, and indeed most conference papers in AI/ML are complete rubbish, because it is trivial to take any architecture and any corresponding benchmark, tweak the architecture slightly and publish a paper. Even dud results are published as "promising". It's just a complete shitshow, and as somebody who is in the field it feels as though you cannot even complain because people get offended. You're just supposed to keep spinning the hamster wheel without posing any hard questions. Moreover, having published at top conferences does not prove that somebody is going to be a good ML Engineer. It just proves that somebody knows how to write a compelling conference paper, which is a completely different skill set altogether.