For a brand new application, is it better to use WPF over Windows Forms? I used Windows Forms before but not much WPF. As far as I know, WPF is the successor to Windows Forms, right?
The application is going to host DirectX Windows (not WPF 3D, but Managed DirectX and SlimDX) with lots of custom controls.
EDIT: The application is a 3D related application, editor, such as modo:
If you're writing lots of custom controls I would choose WPF. Part of WPF's design is to be extendable, composable and the result is that it's dirt simple to write custom controls.
I've done a bit of custom control writing in WinForms and it can be very challanging at time. Layout will really take some time away from you. Several weekends of my life were paid to WinForm Control Layout. Writing the equivalent controls in WPF was a cinch.
----EDIT 2 -----
Given that you're trying to make an editor like the one you showed, I'd recommend going WPF even more. My current project has many features along those lines, as well, and we've decided that the ability to composite WPF with Direct3D content is extremely powerful. It's nice to be able to render your scene into anything - not just a rectangular window. In WinForms, you pretty much were limited to one rectangle, and you had issues with airspace there, too (subtle, but things like flickering issues when menus pull over your hwnd, etc). The WPF compositor with D3DImage gets rid of all of those issues, and lets you use your imagination to construct a very flexible UI. Things like rendering your scene in realtime on the side of a WPF3D object are possible, or using WPF controls directly on top of your d3d scene instead of trying to do the GUI in D3D, etc.
-----Original---------
If you're going to be hosting DX, you might want to consider it - especially since it gives you the ability to do scene composition with your UI and no airspace issues if you use D3DImage.
This does work with SlimDX and WPF.
----EDIT-----
For more information on the disadvantages of using Direct3D with Winforms, and the advantages of WPF/DX integration, see:
MSDN Article on Airspace
Codeproject arcticle on intro to D3DImage
In this case, I'd say follow the lead of your Vendor (i.e. Microsoft). Their own internal production increasingly features WPF; and it's scope (think Silverlight) and depth are clearly greater than WinForms. They also use it increasingly with reference to Best Practices. From all the evidence (here and elsewhere), it appears to be ready for Prime Time developement.
You have epressed no particular affinity for WinForms (I would guess what there is is based on familiarity). You'll need to switch eventually. A new project would be a great place to start the process, everything else being equal.
We dealt with this question about 9 months ago. We decided to go with WPF and so far we're happy with the decision. Yes, there is a learning curve. It's fairly considerable especially coming from WinForms where you have so much to unlearn. I also recommend you have access to a designer otherwise your application will probably look a bit shabby. Also be prepared for some WPF gotchas that will have you spending hours scratching your head saying 'why was this so hard'.
But WPF is a step ahead. The data binding, templating and complete control of how you want your windows to look makes you think that this how WinForms should've been originally.
Oh yes, and be prepared to shell out a couple of dollars for some missing controls. There are a couple of things missing like a Date picker and having checkboxes on tree controls (you can actually template this out, but it's not as simple as winforms in that regard). 3.5 SP1 includes a grid control now, thankfully.
I'm sure I'm missing a bunch more, but that's what I can come up with off the top of my head.
Good luck!