share
Stack OverflowIs LINQ banned in your company?
[+76] [12] Jon Preece
[2010-10-28 16:08:17]
[ c# linq performance ]
[ http://stackoverflow.com/questions/4044823] [DELETED]

I work for a large company who develop enterprise applications which are performance oriented. Virtually every line of code is closely scrutinised and optimized as much as possible to ensure the best performance.

Company policy dictates that LINQ is strictly banned. This is because it is believed that LINQ has a negative performance impact compared to more traditional coding practices.

Is LINQ banned in your company? If yes, what are the reason given for this? If not, what applications does your company develop and why is performance not so crucial?

(35) Some senior architect is scared of getting FP in their OOP? - jball
(17) Your last sentence begs the question. - Michael Petrotta
@jball I think the general belief is that LINQ only makes code look pretty, and that performance is sacrificed as a result - Jon Preece
(2) Where is this (mis)conception of LINQ + poor performance coming from? It's a set of language extensions with the goal of helping the developer write code faster (Where(), Contains(), etc). What's more expensive - CPU cycles, or developer time (dev, maintenance)? Suggest the rule-makers don't understand LINQ. - p.campbell
(10) @Jon Preece - unsurprisingly, using LINQ often allows the compiler to generate optimizations that we would miss if we tried to code query solutions by hand. for loops also make coode look pretty compared to hand-looping with goto, and it also allows compilers to optimize such loops more predictably and fully than they necessarily could with gotos. Seems to me LINQ is in the same boat. - jball
Have fun with SQL injections. At least one dev will eventually not parameterize properly... - Mike S
(63) By this logic, the developers in your company should be coding in machine language. - Jason
(28) I'd leave any company that banned LINQ, at least in the early phases of development, wasting my time on early optimizations. It is easier to get it right with the expressiveness of LINQ. Get the tests passing, then you can safely refactor for performance where necessary. - Jay
(3) I would hate to work for a company where they dictate such stupid restrictions to the developers... People who think Linq is slow just don't understand it. - Thomas Levesque
(2) Community wiki :) - Roy Tinker
(7) From the sounds of it, I'm surprised your company decided to use C# in the first place. Wouldn't C be more performant? (playing devil's advocate here) - StriplingWarrior
(2) I saw the question, just about choked on my coffee, and now after reading I think I will down vote you unless you smack some sense into your company's head - burnt_hand
@StriplingWarrior: Please see my previous comment. You didn't carry that chain of logic far enough. - Jason
(3) Banning anything absolutely seems stupid. I can believe LINQ may not be as optimized in every situation, but do you need every single situation to be optimized completely? Out of the hundreds of things an application does, few are probably used with enough frequency that this matters at all. Why spend effort optimizing anything that's not a bottleneck? Why not select the tool/technique that fits the job: longhand where performance is important, ease of coding where its not? - jamietre
(1) Perhaps your management is not differentiating between LINQ to SQL and using LINQ statements in your code to get rid of for loops and the like. LINQ to SQL creates garbage and is slow and should not be used. - hyprsleepy
"I think the general belief is that LINQ only makes code look pretty, and that performance is sacrificed as a result" that's not grounded in reality. I'm not even sure where such an idea would come from; there's nothing inherent to LINQ that suggests anything of the sort. - Rex M
(3) @hyprsleepy that's a bold blanket statement to make. SO runs on LINQ to SQL and is wildly successful. Updates and deletes are where it has issues but there are work-arounds for that. Without knowledge of the ORM and understanding how to configure it properly you can't expect optimum performance by default. - Ahmad Mageed
(4) Anyone that bans a useful language feature outright for 'performance' reasons (without measuring specific cases) and who is NOT coding 100% of their code in pure, optimized Assembler is a hypocrite in my book. Just my 2 cents :) - Michael Stum
Snails should have been made extinct because they carry their houses and move so slowly as a result. - BoltClock
(1) @Ahmad Mageed - It is not, you can back it up with facts and investigate what Visual Studio is doing behind the covers like this guy, chrisbrandsma.com/2007/08/…. Also there have been speed tests that conclude the same thing such as this one peterkellner.net/2009/05/06/… or this one social.msdn.microsoft.com/forums/en-US/linqprojectgeneral/…. - hyprsleepy
(1) Here is another speed test: codeproject.com/KB/cs/linqsql.aspx - hyprsleepy
(2) @Ahmad Mageed, except LINQ to SQL is not an ORM and Microsoft themselves don't recommend to use it. In case of SO it may be fine, since select queries can be precompiled, but for scenarios where you update more often than you select LINQ to SQL is going to cripple everything. - Egor Pavlikhin
Shouldn't you ask this at programmers.stackexchange.com? - Graviton
I used to think the same way. But as my projects became more ambitious, I slowly started reinventing LINQ anyway, or at least the extension methods and a query provider. When I realized what was happening, I became a true believer in LINQ. - hoodaticus
[+160] [2010-10-28 16:17:42] Reed Copsey [ACCEPTED]

I, personally, do not ban LINQ usage in my company - but rather encourage its use whenever appropriate. LINQ is very expressive, and I find that it produces code that is often much more maintainable, and higher quality, than "traditional" methods of development.

In addition, I often find that many developers tend to write more performant code using LINQ than "traditional" looping methods. The streaming nature of LINQ can dramatically cut down on the runtimes if used correctly.

Personally, I find that any company that "bans" the usage of a specific language or framework feature outright has higher level problems. Every technology exists for a reason - and every feature has its place. Whether it's appropriate in a specific scenario is another issue - but banning it completely is a sign of poor management of developers, in my opinion.

If not, what applications does your company develop and why is performance not so crucial?

Performance is absolutely critical to me. My company develops scientific software, and many of our routines have runtimes in the hours (or even days), so every ounce of performance we can squeeze out is very useful.

That being said, performance doesn't come from micro-optimizing in most cases - it comes from designing a better architecture and "large scale" algorithms. The key for this to be truly successful is having a good design, and keeping the code as simple as possible. Simplicity helps in profiling tremendously - which in turn can pin point the real performance issues in the application.

Occasionally this will be due to a LINQ statement, but very rarely. More often this is due to a poor design decision. I find that LINQ actually reduces the frequency of poor design decisions. It's much easier to refactor a LINQ statement into a more performant tight loop when necessary than it is to try to profile an application that's "chattier" in terms of code than necessary.

In addition, LINQ has some huge performance opportunities, even on a small scale. For example, it's much simpler to parallelize a LINQ query via PLINQ than to try to parallelize many other constructs, for example. Granted, it's not magic, and care still needs to be taken, but LINQ tends to parallelize with fewer issues than loops. If performance is the goal, simple, clean code should be the target, and LINQ helps achieve that more quickly.


(1) +1 Reed. Bang on, IMO. - p.campbell
+1. Hear, hear! - Roy Tinker
Couldn't have said it better myself. I've voted for a few answers, but this is the one that should be chosen as "the" answer. - StriplingWarrior
(33) Right on, Reed. You can spend a little time making a working program fast, or you can spend the rest of your career trying to make a fast program work. - Jim Mischel
(11) ++ Hooray for pragmatism! SO is full of questions where people think that you get performance by examining code. You get performance by a) keeping things simple, and b) finding out where the problems are. Here's my example: stackoverflow.com/questions/926266/… - Mike Dunlavey
People simply need to be aware of LINQ's performance characteristics--that if you scan a list with LINQ-to-objects, it is O(N), just like a for-loop is. In very hot spots I would switch to a for loop, which avoids the overhead of lambda and interface calls. On the other hand LINQ is lazy, which can sometimes make it faster than a for-loop despite this overhead. So banning LINQ everywhere is just stupid. - Qwertie
(13) The fastest code is the code that doesn't execute because you found a way to avoid writing it. Simplicity is O(0) - slf
(3) "Personally, I find that any company that "bans" the usage of a specific language or framework feature outright has higher level problems" Yes. This x 1000. - matt b
1
[+54] [2010-10-28 16:12:23] Jon Skeet

why is performance not so crucial

Performance is often important - but that doesn't mean that every line has to be optimized to the hilt. Heck, often whole components are far from the critical path.

Often it's more important to get the overall architecture/design right - that's where big savings can usually be made. It's easier to spend time on the higher level design if you know you'll then be able to implement it in a readable, maintainable way... and LINQ can help you to achieve exactly that sort of implementation.


(3) My thoughts exactly... why spent a lot of time achieving optimal performance where the overall benefit is irrelevant? In most apps, I would guess on the order of 5% of the code is responsible for 95% of the resource use. Focus your effort on writing slick code where it matters, use higher level tools or lazier code where it doesn't. - jamietre
2
[+20] [2010-10-28 16:58:00] StriplingWarrior

I ran across a hand-coded for loop in our system a couple of months ago, which was commented to indicate that it was done this way because it would be faster than using LINQ. At the time, this developer didn't fully understand the inner workings of LINQ, and made the erroneous assumption that performing multiple operations in a single loop would be more performant than chaining LINQ operations.

I suspected that this wouldn't actually save us much time, if any, so I wrote a quick benchmark which found that a one-line LINQ statement actually evaluated faster than the loop. I showed it to the developer (who has no ego, and immediately saw the error in his ways), and we all moved on as better people.

There are times where LINQ can be misused in such a way as to be costly, performance-wise. But most of these mistakes can be caught with a code review (which it sounds like you already do religiously), combined with maybe a few brown-bag lunches to discuss pitfalls to avoid.

One other thing I'd point out is that I was able to get a 3x performance boost from some of our longer-running unit tests simply by adding .AsParallel() to the beginning of a LINQ query and changing a foreach statement to items.ForAll(...). PLINQ makes it really easy to get a big performance boost where you probably wouldn't otherwise have had time to make a multi-threading optimization.


(3) +1 for evidence-based programming. We'll turn this thing into a science eventually ;) - Ranieri
(3) +1 for your fellow "no ego developer" - blizpasta
3
[+16] [2010-10-28 16:15:51] Bob Jarvis

At the company where I work the rules are quite simple:

  1. Everything not approved is forbidden.
  2. Everthing which is approved is mandatory.

LINQ, of course, is not approved.

Power to the proletariat. :\


(3) i like this answer :) - Jon Preece
(1) Thank you, comrade Preece. :-) - Bob Jarvis
Wow. Not much room for one's professional opinion there. - Roy Tinker
(3) @Roy: just life in the Land of Corporate IT. - Bob Jarvis
(4) As it should be. Otherwise 10 professionals mean a mess of standards. Been in a project like that - over 10 years they got around 15 different data access approaches INTO ONE APPLICATION because every module developer was allowed to voice his professional opinion. The result was impossible to maintain. - TomTom
(2) Double plus good. - StriplingWarrior
(2) @Tom: Not necessarily. Good professional engineers know better. - Roy Tinker
(6) @TomTom: Do you not see any option for a middle ground? Something between a free-for-all and complete inflexibility? - Jon Skeet
(2) @Bob: time to start looking for a better job? - Roy Tinker
I say sneak it in! Proof is in the pudding - burnt_hand
(1) @Roy: you hiring? :-) - Bob Jarvis
(7) @burnt_hand: on one hand I'd love to do that. On the other hand, I agree with @TomTom that this can lead to confusion. Gripping hand, we've had people walked out the door for doing such things. I've got a wife, three kids, and various livestock, ALL OF WHOM like to eat reg'lur and have a roof over their heads. Getting booted just because I Know Better (regarding software development tools - which of course I do :-) is NOT on my list of things to do this week, thanks muchly. :-) - Bob Jarvis
(1) This is why I left a big corporation for a smaller more agile one. Give me room to breath! - James
@bob jarvis - they say you should do everything you can before you get old(er) and become responsible for others. Reckless bleeding edge coding, I guess, is one of those things - burnt_hand
4
[+8] [2010-10-28 17:37:15] David B

Virtually every line of code is closely scrutinised and optimized as much as possible to ensure the best performance.

Scrutinize and optimize this quoted line, and realize that greedy optimization can't achieve the gains that a creative thinker can.

There are mainstream professional programmers (who do write data acess code) out there that do not comprehend what an index is, what a query optimizer does or what a good query plan looks like. These programmers do have to optimize every "line" because their data access runs 1000s of times longer than it should.

Is LINQ banned in your company?

No.

If not, what applications does your company develop and why is performance not so crucial?

I use LinqToSql to write more performant data access than I could write by stored proc or by raw ADO.NET.

  • Programmer brain bandwidth matters.
  • Minimizing redundant access across many operations matters.
  • Profiling tools matter.

5
[+6] [2010-10-28 16:35:16] Jim Mischel

Company policy dictates that LINQ is strictly banned. This is because it is believed that LINQ has a negative performance impact compared to more traditional coding practices

We don't ban LINQ simply because we don't share those beliefs. Instead, we profiled some representative samples using LINQ and using more "traditional" coding practices and found that the performance of LINQ was in line with the performance of our hand-written loops.

In the work we do, getting something working is paramount. We've found that LINQ lets us write working code more quickly, more clearly, and in fewer lines. In the very few cases where a LINQ expression appears to be a performance problem, we've found that the problem isn't with LINQ but rather with our overall algorithm. Hand coding an alternative to the LINQ expression almost never results in a significant performance boost.

Your last question is "why is performance not so critical?" Believe me, performance is critical in what we do, and we've found that using LINQ does not negatively impact performance.

I would suggest that you question the reasons behind the LINQ ban, and perhaps do some profiling of your own. You might be very surprised at the results.


6
[+5] [2010-10-28 16:17:30] Steve Haigh

We certainly don't ban LINQ, I use it a lot, but in perf hot spots we have in one case I know of gone back and reworked LINQ code to non-LINQ. Performance is extremely important in the applications I develop.

Banning a technology as powerful and as useful as LINQ is very short sighted and is not the way to improve performance. If you want to imporve performance then invest in a good profiler and take the time to use it.


7
[+4] [2010-10-28 16:11:53] TomTom

LINQ is not banned. Low quality O/R mappers (LINQ2SQL, Entity Framework) are. We use alterantives - NHibernate when a full ORM is needed, BLToolkit for fast mapping to SQL. Main application is very real time sensitive, but LINQ (language extension) has a lot to offer, and the impact is neglegible if well used (reviews).

To add to that - I am now mostly in the financial area. Due to the nature of the application, elemets like "change tracking" are not needed. There ARE NO CHANGES.

Every entity written to the db or pulled from it is immutable. Changes are done by submitting change requests to a service, which will do the calculations and write a new line to the database. There are "Entities" that contain the key of the item and the last known data value as single property (as published - pulled from the db), but it is, as I said, immutable.

Performacne is crucial because the mechanism is event driven and has to cale to a million events - per second. THis is why we go for the db operations now there with something "thin" like BLToolkit - we dont need any of the higher up funcitons.


(7) Curious what's "Low quality"? - p.campbell
(1) In my experience, LINQ to SQL has been very useful -- I wouldn't call it "low quality". It translates my LINQ expressions to good SQL code. - Roy Tinker
The problem with it is that it tries being an ORM without doing about 85% of the stuff an ORM is supposed to do - and was supposed to do in the last 15 years. NHibernate is still running circles around it. THe problems are QUITE many - too many to talk about here ,but a google search will find you some nice documents. - TomTom
(7) @TomTom: It depends on what the task is. For some applications LINQ to SQL may well be absolutely fine. For others it won't be up to the job. Ditto the Entity Framework. It doesn't have to be "one size fits all". - Jon Skeet
(1) I'm curious, what is the 85% of stuff that LINQ2SQL is missing as compared to NHibernate that you simply can't live without? We're using LINQ2SQL at my company rather easily. I can't imagine whatever OR/M features that we're really at a loss for. If you mean silly things like lazy-loaded collections or whatever, then I don't want those things anyway. All queries should be explicit in my opinion, not a fly-by-night sort of thing with lazy-loaded collections. - James Dunne
Remoting, cross transaction caching, streaming results for queries that just return many many items are only examples. Lazy collection are critical in most end user scenarios. If you think LINQ is bad you should see our current approach - you send a query rexwust object and get results streamed via packets for as long as it takes (responses CAN have some hundred thousand items returning for visualization). - TomTom
8
[+1] [2010-10-28 16:36:41] Snarfblatt

I thankfully work for a company that isn't afraid to let their developers think. An outright ban seems rather short sighted.


9
[0] [2010-10-28 16:49:43] arlen

When you start using LINQ, It's gonna be hard for you to switch back to traditional one. I guess there are bunch of doubts that keeps the company away of using LINQ. Cause there Is a little bit chance that Microsoft will remove LINQ in the feature, and they not gonna support anymore. and the other one Is your upper level managers don't know about the LINQ and new features and they don't wanna investigate more. They just say don't use that.


While LINQ to SQL could be deprecated someday in favor of Entity Framework, Microsoft has a huge investment in LINQ. MS preserves backward compatibility 99% of the time, and C# has built-in language support for LINQ. At a minimum, LINQ to Objects isn't going away anytime soon. - TrueWill
10
[0] [2010-10-28 17:49:55] James

Granted LinqToSql has some memory overhead with the DataContext. But it does give you the power to override any table operation (even select) with a nicely tuned stored procedure. This gives you the best of both worlds by letting you tune where it matters. LinqToSql is quite powerful and much higher quality than some others on this thread would have you believe.

When you calculate the cost of doing it the old way compared with the new way it should not be too difficult to sell.


11
[0] [2010-11-22 18:22:55] Lone Coder

Linq is not banned at my company, but SQL Server essentially is. We evolved out of the LAMP stack and are still using MySQL on all our servers and haven't felt a need to switch to MS SQL.

Because LINQ to sql is exclusive to MS SQL Server we haven't been able to use it. So that part has essentially been baned. I hear that LINQ to Entites works with other databases but the MySQL drivers weren't ready for it last time I looked.


Linq can be used without a database - Lavinski
12