This one is for any software developers out there who have worked on both front end UI type code and automated back end code ("Back end" does not mean the server side of the CRUD- more the transformation of financial data and applying of complex business rules by automated processes). There's a LOT behind this question- I original typed it all out but it ended up being VERY long winded and tl;dr. So let's start general- I'd love any feedback I can get, as the project I am on is really struggling.

Our specific struggle is that our back end code (the part that handles the processing of complex financial transactions) is coming along great, while our front end UI code (CRUD with a lot of complex business rules regarding which users can do what) is constantly riddled with defects and is consistently responsible for pushing back our user demos.

I've always thought that back end, complex financial business logic was more difficult to write well, especially when you consider it has to be highly available and extremely stable, but I am starting to feel that while it is certainly more critical and complex, it is also fairly deterministic, whereas UI code tends to be filled with side effects. It seems that the UI code might in fact be more difficult to write.

So in your experience, is UI code more difficult to write well, or is writing automated back end processes the greater challenge? Or if there is more nuance to this question (which there certainly is) what factors come into play in answering this question? The real result I am looking for is what steps we should take to alleviate the struggles the UI team is having. I am on the back end team and there has been at least passing mention of switching up the team makeup.

The back end stuff is frequently harder to validate in quality assurance. It's possible that you don't know just how unstable the back end stuff is until after the front end is working perfectly.

Hopefully you've got some unit testing of the back end that doesn't require a front-end UI to test.

User Interface is more time consuming when you have complex business logic in my experience. GUIs can be easy if your program is simply displaying data and asking the user simple questions. But when it gets down to "user can click on this, but not if this is checked" or "you must validate this input based on this complex business logic" that tends to be more time consuming in my experience. Then there's the whole GUI design aspect. Who is designing the GUI? Do they have a clear design when they start or are they making up the design (and changing it) as they go?

I find that this is the biggest concern, personally.

Good programmers are not ever necessarily good UI designers. You need to have someone sit down with the users, or, preferably, an internal user proxy who understands the business needs well, and design a UI, without any regard to what the back end looks like.

When integrating the two, you may find inconsistencies, and while this might be a real problem, more often than not, it's a miscommunication between the users and the developers, and can actually help to find "real" issues.

Our back end stuff is 95% covered by unit tests (only things that aren't are integration points such as web service calls and configuration wire up code like IoC bindings). We are a small team of two developers and both of us are big believers in TDD, so basically any code that contains logic is test driven.

Byond that, we have an application we wrote to test our basic processing engine by running it in multiple threads (the processing must be able to synchronize across multiple threads and machines for effeciency/high availability), throwing a ton of data at it, and verifying no processing collisions occur/exceptions are raised.

Finally, we have two full time QA who are dedicated to testing our stuff by looking at database records and mock data input and output.

Obviously we still have to make sure our stuff integrates well with the other team, but I'm very confident in the quality of what our code is doing. If issues do show up as the front end comes along, I think we'll be able to correct them fairly quickly.

To contrast, the UI team has something like 30% test coverage (and I probably wrote 30% of those tests before I was moved off to work on back end stuff). I realize that test coverage is easier/more benificial for back end, so I really am trying to be open minded and not pull the "if you just did things my way, you wouldn't be having all of these issues".



This is exactly the kind of stuff they ares struggling with. This app will be used by many different financial orginations, each with its own structure, so creating a UI that is easy to use but handles all the scenerios can be challenging indeed.



We do have a designer + 2 business analysts and the UI team is working off of wire frames, so there is clear direction. That being said, we are demoing changes every three weeks and obviously the customer is going to have input on what they see (even if they've agreed with the wire frames- once they actually see it it's a whole new ball game).

Of course this affects the front end WAY more than the back end (the users don't really care about back end stuff- it just has to work), but our pain point isn't customer satisfaction with work- it's that at the end of every iteration we have dozens of major defects that are causing us to push back demos, which obviously looks very bad to the customer.

Fortunatly, this application is a re-write of one that existed before (At another company). Unfortunatly, the actual UI coders were not involved in the previous project. Fortunatly, the BA and end user were, so they have a VERY clear concept of what they want (and what was a pain point in the UI before). By default, our goal is to keep the UI the same, but obviously a lot is changing, espcially since we have a designer who is making improvements to what we had before (some VERY good improvements).

Basically, the good thing is that developers are not really responsible for the layout/functionality of the UI. I pushed VERY hard at the beginning to get the designer added to the team, and I'm glad I did. It's a really good thing to have someone who is not a developer be responsible for the UI layout and behavior.

Btw, I really do appreciate all the comments. I know that some big decisions are going to be made soon and I will have some influence- I want as much objectivity as possible.

I'd say your problem is test coverage in the UI side of things. It sounds like you have the classic problem of one thing being changed intentionally affecting other unforeseen parts of the app. Just because there isn't Serious Business Logic in the front end doesn't mean it doesn't need to be tested.

If I were joining your project with the goal of attaining a sustainable and predictable pace, I would try and combine the front and back end teams into one team with ownership of the whole code base. If the response is "well, the front end guys aren't as strong developers as the back end guys" then perhaps you've found your answer as to why you're having regressions only in the front end. The answer isn't necessarily to fire the front end guys, but is more likely to be to shuffle everyone, and get the front end guys understanding what the back end guys do, and vice versa.

The first order of business would then be to get the front end tested as well as the backend. In my Ruby on Rails world, this would be Selenium for browser testing, Jasmine for javascript testing, Cucumber and Webrat for story level testing, and plain rspec for the controller and model level code. Don't be afraid to slow down and figure out what works for you - any code that's worth writing is worth test driving.

This in fact happens a lot- QA reports a bug- they fix it, and it breaks something else. Of course my first response is like yours- "if there were tests around that functionality, then we wouldn't have this problem". However, using TDD is being seen as a style difference, and "it's hard to teach experienced developers new tricks". And obviously TDD is not that same as test coverage, but it is how I (and a few others on the team) achieve good test coverage.

Hmmm- this is interesting, as I was the one who suggested the more specialized approach. The reason being is that rather than having developers need to understand understand every piece of architecture within the app, they can focus on one area.

That is definitely NOT the response, at least from the project managers. They view all the developers as equally strong. QA is starting to grumble a bit about the other team and I am trying to keep them calm, but they are getting quite frustrated.

This point is why I kind of made this post. One of the project managers asked me "Is it more difficult to work in the front end?" My answer was "if it is, it is because we made it that way- front end code is not inherently more difficult than back end code". After telling her that, I started to question myself- hence this post. My belief (open to being challenged) is that the abstractions we created to solve our front end problems are not as good as the abstractions we are using in the back end, so now working in the back end is indeed easier. But of course, that WOULD lead back to developer skill, as we are the ones who created these abstractions.

The front end is Silverlight (I know, I know- not the issue here!), so anything in C# is easily testable using the same testing framework we are using for the back end (the app is C# through and through). The problem is that unit testing is not valued by everyone- it seems to be one of those "if it works for you" and "nice to have" kind of things.

If they're seeing bugs -- let alone regressions -- that unit testing would have caught, and that in itself isn't scaring them into writing more unit tests, then I wouldn't call them all that experienced. TDD as such could indeed count as a difference in development style, but requiring tests for such functionality as is testable, and not marking the functionality complete until the test is complete, isn't a style thing, it's a "this is what the quality bar is for check-in" thing. Which is within a project technical lead's mandate.


Yes. Many software bugs are in fact composed of two bugs -- the bug itself, and the bug whereby the test suite didn't find it. There's little point fixing the first without fixing the second while you're there. Of course some projects -- some environments -- are more amenable to testing than others, but it sounds like your one is relatively lucky on that front.

Peter

I agree 100%- and we've had a lot of discussion along this point (myself, the project managers, and our two technical leads)- I actually don't care if people use TDD- I DO care if the code is cleanly written, refactored, and unit tested. To me, that's just doing your job. You aren't doing your job if what you are checking in isn't quality, and that includes unit testing. The issue is, how do you enforce this? We've tried emphasizing the importance of unit testing, but at the end of the day, what do you do when people just don't do it? It's not so much that they are defiant, as they always have excuses.

I want to point to the success that we TDDers are having and say "look- this works!", but the problem is that people say "yeah, but what you're doing is easier to write tests around".

Really that's a workplace psychology question, not a technical question, so as a techie I'm no expert. If someone started checking crap into an open-source project I was maintainer of, I'd revert the commit for a "first offence", withdraw write access (in favour of submit-patches-via-intermediary) on a "second offence". In the workplace, though, with people you have to chat to round the watercooler, it's probably better to be less hard-nosed in the pursuit of quality.

One way to be hard-nosed without seeming so, is to invent "staging" and "stable" areas, where people can check in testless code to "staging", but it needs tests before it's promoted to the "stable" branch where releases are made.

Peter

I guess I was veering off topic a bit- I think the thing is, we can be hard nosed about this stuff, but if we are then we need to be VERY certain that we are "right". And if the reason the other team appears to be struggling is simply that they have a more difficult task, well then pushing them won't really have a positive effect.

It seems, though (based on only the limited information I've provided you guys), the sentiment here agrees that lack of unit tests is an issue (as I've thought) and this is something we should be willing to push on. I'm also thinking (based on this thread) about what doing away with the Front End vs Back End designations might produce and seeing if that might be something to try as well. I am hesitant on this because we ARE getting a big boost of speed by having the two of us focusing on back end stuff- we are blazing through it MUCH faster than anyone else would just because of our familiarity with the business domain (he and I worked on a system with the same business logic at a previous company). That being said, it is difficult to "lead by example" if we are completely cut off from the development work the other team is doing.

Maybe I'm misunderstanding your problem, but how do you do unit testing on a UI? I'd be inclined to think that TDD doesn't work all that well with UI code.

We use a MVVM model for our UIs where our "views" (pure xaml) have no code in the codebehind and use databinding to connect to viewmodels, which provide data and respond to events triggered by the views. We do not put unit tests around the xaml (though this is possible in Silverlight), but it is easy to put unit tests around the viewmodel code. That is, "when x button is pressed xyz request must be sent to the server" is a fairly trivial test to write.

I have not written much javascript, but I would think that tools exist to make similar techniques work with well designed HTML/JS UIs as well.

Additionally the UI team is responsible for writing the server side code that sits behind web services and performs all of the CRUD logic, including validation. This code is easily testable.

So there is quite a lot of testing that can be performed, though obviously they'll never completely match the test coverage rates that we do.

(IANA professional computer programmer, but I am an expert on everything, or so that voice inside my head keeps telling me.)

Honestly, I think of UI as the most complicated part of a computer program. Backend stuff is far more regular than any UI of any complexity could ever hope to be. It tends to be easy to model, whereas UI stuff has to deal with the vagaries of human psychology. I also tend to think of UIs as needing to be open-ended, such that it is feasible for the user to do things that the developer might never have considered.

That said, it sounds like your UI might be the opposite of that, and just consist of parameterized views into the existing business logic in the backend. Maybe I'm wrong.

My vision of a UI is complex to develop and difficult to test, in which case your assessment that the UI might be more difficult to develop is probably right, but your solution of requiring more unit testing might be wrong.

On the other hand, if it's a merely presentational UI, then it's probably not that complex to develop and likely pretty easy to test, but that makes your assessment of the difficulty of the task suspect, in which case, requiring unit testing might help, but is probably not the ultimate problem.

What I guess I'm ultimately getting at is that, as an outsider, I'm inclined to disagree with your take on the conclusion here, and say that the two most likely scenarios in my mind are that either unit testing is probably not very productive, or that the development group is not very competent. (I'll qualify that last speculation by saying that it might not reflect on the competence of the individual developers, but might be a political, cultural, or managerial issue. Then again, they might well just be incompetent.)

I'd suggest that you take a closer look at their current testing methodology, and instead of coming at it with your own prejudices and seeing it as the wrong kind of testing, try to evaluate it within its own idiom. I'll bet you'll find that it's lacking within its own internal logic. And getting them to improve or repair what they're already doing is likely to be far easier than trying to get them to change their whole procedure. (Then again, changing their whole procedure might be more productive long-term as you prevent them from falling into old habits.)

Basically, I'm bored and killing time. Ignore me. I don't know what I'm talking about.

Edit: Ok, so I had a big response- but I think some of it was a bit whiny and unproductive.

Suffice to say- I am always interested to hear your opinion Bitt, and I have a lot to think about.

I'd argue that erasing the division between front and back end developers will make the question of "which is harder" irrelevant. It may slow you down at first, as people get up to speed with the parts they're not familiar with.

I'm of the opinion that by far the best way to accomplish this is by pairing. Start with asking them to help you on something you'd normally work on. Show them how a failing test makes the job easier. Once you've got some buy in, try following the feature through the front end, all the while using tests to "prove to yourself" what you're doing is working.

There a nice video from Velocity on creating cultural change that's worth watching, even if not directly relevant. If at all possible get people to join you, don't tell them what they should be doing.

As to front end testing being difficult, there is a certain class of combinatorics that is difficult to test exhaustively. However, it's not hard to test that each validation error displays properly, and that multiple validation errors display properly. That's usually enough to convince a sane person that any combination of errors are going to display properly. (Assuming a reasonable developer who's planned for the errors area to grow with the list of errors)

Are there really UIs that allow users to do things never imagined by the developer? I'm having a hard time thinking of one. Photoshop has a very complex UI, but each action generates transform that is layered onto the image. Automator lets you do things never imagined by the developer, but in reality it's just a series of pipes between actions. The UI lets to pick actions and arrange pipes.

I'd argue (though perhaps be wrong) that getting beyond what's been thought of by the developer requires code or at least a domain specific language, not a UI.

Of course there is, we come across it all the time, in one of our products (CAD), somebody may have something selected in the diagram, there are modifier keys which apply temporary transforms while held down, it's possible the user selects some random toolbar option or menu option which given the complexity of the program fails to correctly apply the transform.

All operations were perfectly valid to "do", but only the combination of them cause the issue. As a program gets bigger and more complex it's easier to miss cases like this - the developer tests the standard UI cases but misses more obscure ones.

s/do things/accomplish tasks/ above.

There are a limitless number of input combinations, and there's potential for those to interact in unforeseen ways. However, the ideas encompassed in drafting using a CAD program were considered by the developer and the UI for doing that was considered by the developer and designer. Perhaps a "better" UI would have indicated which features were compatible with the active modifiers in the UI.

There certainly are bugs that will not be caught no matter how much manual or automated testing is done. Front end code has more entropy in its inputs and perhaps will have to deal with more unforeseen circumstances.

So here is my personal take on UI development- and I'll be the first to admit that what works for me may not work for everyone else.

I have always struggled with UIs because their even driven nature makes them feel very non-deterministic. This is what you guys have been talking about in the last few posts, and it's kind of a point I alluded to in my first post. It's just so hard to think of everything a user can do at any given point in time. Now I'll say, before I go to far, I work on line of business apps, in which we are going to be dealing mostly with CRUD type operations, though obviously with rich a user experience as we can build. So what works for this type of app probably is not a good answer for a CAD type application.

When I first started doing TDD I tried to apply the technique to UIs. Eventually this led me to the previously mentioned MVVM pattern (though I didn't know that's what it was at the time). My feeling at the time was that it was very tedious work to try and test drive a UI, and there didn't seem to be much benefit. This was until I released the code to QA. There was such a noticeable quality difference in what the TDD approach netted me vs my old coding that QA actually told the project manager "whatever is different about the development, make sure they keep doing it".

So I stepped back and took a look at what was different, and essentially it was this: Two side effects of test first development was that I was forced to wrap the "state" information of my UIs into separate classes (which would become "viewmodels") and then handle modifications of this state in very small chunks as I developed (because of the nature of writing small, focused tests). What was happening is that I was FAR more considerate of what any effect any given action might have on the entire state of the viewmodel. So when I had to handle the action "user pushed button x", I had right in front of me all of the potential state changes, and I started writing small tests around what would happen to each one after any given action.

At the end of the day- this just pointed out to me that I was not handling the complexity of UIs well before- and that TDD was forcing me to start writing better code. It was a crutch, in fact. Since then, I believe I have become a better coder of UIs even if I'm forced into a situation where TDD doesn't work because I have developed a better skill at breaking down the effects of total state of any given input a user may perform. Of course, having tests around this kind of stuff is nice as well, but I definitely became better at writing UIs thanks to TDD.

So that's what worked for me- but being as a lot of what I gained through applying unit tests to UIs was a "side effect" of a particular TDD approach and really just an improvement to poor coding on my part, I cannot say that others will get the same quality boost applying TDD, or even unit testing, to UIs.

The only reason I'm asking the question in the first place is to pin point why the UI team is struggling so much. A project manager directly asked me whether they should be having a harder time of it due to working in the front end.

I'm a fan of pairing, and this is something we did a lot of at the beginning of the project. I have actually paired with each person on the team (except for one guy we added last week). Unfortunately, a lot of the early pairing devolved into arguments over "style" and productivity really struggled. I still tried hard to pair as much as possible until the teams split in effort to get everyone on the same page.

After the teams split, though, pairing virtually stopped. For me I have stopped pairing because the other guy on my team and I have worked together for a couple of years now and both write very similar code. We communicate regularly on development questions (we work in the same room) and review each others code quite a bit. Pairing didn't seem to give us the value it has in the past (we have paired in the past when we first started working together).

The other team has stopped pairing as well- I think mainly due to speed pressures.

I think we had to split into sub teams, though, because with QA, BAs, and everyone else we are currently at 20 people on the overall "team"- this makes for very painful stand ups and it is easy to get lost in the mix.

I agree 100% about creating cultural change; however, this is where I have to note that cultural change is going to be valued less and less as we approach the "end" of the project. That is, the team is currently being augmented by several contractors that are all going to be gone in about six months. At that point we will scale down to three developers and an architect, all who have worked together and share a similar culture (big fans of TDD, etc.). So changing the habits of temporary employees doesn't seem worth expending too much effort. That being said, we still have to make sure we hit our immediate deadlines and end up with the quality we want.

I'll still watch your link though when I get a chance.