Making Dot plot

Mar 21, 2013 at 10:21 AM
Edited Mar 21, 2013 at 11:18 AM
I want to make a Dot plot program ( specific Dot-plot of bioinformatic) in which two complete genome can be aligned. The GUI of that dot-plot should be rich enough that after alignment specific region on dot plot can be selected ( regions like palindromic line, mini-satellites, conserved regions etc) and sequence of these regions can also be retrieved. After retrieval of sequence, they will be BLAST and functionally annotated. (I can do this post-retrieval part, but I mentioned here for describing my full scenario)

Actually I am new to this library and want to know whether this can be done. Describing overall picture briefly would be much helpful for me and I am intended to do it on WinForms. Thanks in advance :)
Mar 21, 2013 at 1:57 PM
I'm very interested in make visualizations tools for comparative genomics like these, i suggest you something more user friendly to the final users, like WPF, ASP.NET and maybe silverlight. I want to discuss the idea please tell me more about your project.

Regards,

Leo
Mar 21, 2013 at 4:24 PM
Edited Mar 21, 2013 at 4:26 PM
This is great to hear. Actually I have expertise on WinForms and ASP.NET. XAML platform is new for me but I think it will not be hard to learn. Actually Dot plot are common now, but I feel there is some space to develop a Dot plot program with more enhanced features. Like selection of region and do automatic annotation. (by sending selected region's sequence to any BLAST or annotating server).

To be honest this is my final year project of my bachelor in Bioinformatics and I want to make the Dot plot with much enhanced UI which enable user to perform the above work.

I did some research on this topic and found these very useful topics through codeplex discussion:

WPF + Prism + Biology!

Dot-plotting with .NET Bio part 1

Dot-plotting with .NET Bio part 2
Coordinator
Mar 21, 2013 at 6:52 PM
Hi,

.NET Bio is certainly capable of managing the data side of what you want, but it includes no specific support for visualizations; for that you need to turn to some other aspect of .NET. I'd recommend WPF for any significant UI work, or Windows Forms for really simple stuff. WinForms isn't nearly as customizable, it's really for traditional LOB desktop apps more than anything. WPF was created for more modern UI uses and to be highly performant and visual. At this point, I'd recommend you avoid Silverlight entirely - it's officially been killed off as a product and is barely being support as far as I can tell. Not only that, but .NET Bio is very limited in that environment anyway so it's not ideal. DeepZoom was very cool, but can be reproduced in the desktop world too and you have far more control, power and memory running in a full desktop app.

As mentioned earlier, HTML5 is certainly an option, but you'd need to push the bioinformatic analysis code into the server-side (ASP.NET) and then push down data to the client to visualize, or use some other bio library that can be used by Javascript.

There are several people on this list who'd be happy to help, or to act as a sounding board for ideas!

mark

Mark Smith

mark@julmar.com | @marksm | 214-774-4749 | www.julmar.com/blog/mark


Developer
Mar 21, 2013 at 7:17 PM
Hi Mark,

Interesting thread. Given that silverlight is dead, do you know if that means that WPF is heading the way of the buffalo as well? The two technologies seemed to overlap quite a bit, and I can't imagine the death of one bodes well for the other.

I am sort of getting the sense that the trend to mobile is pushing good available options away from desktop development and towards technology that relies more on a server/client programming like html5/javascript. It seems that the server pushes data to client dynamic doesn't work well if using .NET since the browsers can't work with C#, which to me suggests that .NET would not be the platform to develop either the client or server side logic (though the other alternatives seem far more annoying than silverlight would have been).

I am far from an expert on recent developments in the software industry, but one of the things I love about c# is how easy it is to code and play with visualizations, and would be interested to know how other C# people are thinking about doing going forward for visualizations if it is being deprecated (or is it?)

-Nigel
Coordinator
Mar 21, 2013 at 7:49 PM
Hi Nigel,

Great comments! First, let me be clear that I am not a MS employee, and cannot speak officially on the company's strategy or decisions - it's all really my observation and opinion. Let me also apologize for the long post!

No, I don't think WPF is going away - I think it's just "complete". In other words, it continues to be maintained and will evolve with .NET over time but Microsoft considers it finished. They still recommend you use it for desktop apps. Silverlight, on the other hand, is actually being discouraged. As to why they did this, I think what you are seeing is a strategy shift. WPF was, and always will be, oriented to the desktop - it never played well in the browser world. Silverlight was intended to fix that - it targeted the web primarily and was cross-platform but also allowed for out-of-browser execution (sort of - the browser was really still there, just hidden). A lot of companies loved that because they could build desktop-like apps using close-to-WPF technology and run it on Macs and Linux without having to rewrite it. But there was still a growing segment that was alienated by Silverlight (and Flash) which was the slate and phone market. I think the development of WinRT and specifically the Windows Phone and Surface devices forced Microsoft to recognize that Apple had made a smart decision not including browser plug-ins like Flash or Silverlight in their mobile strategy. The lower-end devices just can't run power hungry apps like that and have any significant battery life. So, in order to embrace this growing market and be more "standards-friendly", Microsoft made a pretty huge shift to say that for web stuff HTML5 is the way to go - it's portable, compatible and (almost) standardized. That left Silverlight in a bad place because it isn't as ubiquitous as HTML5 for the web apps, and not as powerful as WPF for the desktop. It was a product with nowhere to go and that's really what killed it off. Instead, Microsoft has gone towards competing directly in the mobile space with Windows 8 - the syntax and tooling we used with Silverlight (and still do with WPF) is now in the Windows Runtime (WinRT).

I agree with you -- the mobile arena seems to be growing and I think a lot of people are replacing traditional computers with slate devices but personally, I think C#, .NET and the desktop will be around for a long time. Slates, while popular, really only excel at information consumption - they are pretty painful for significant content generation. Surface might change that - the Pro model is actually pretty nice, but it's a bit heavy and still has some rough edges to really replace my laptop at this point. For the kind of things people on this list want to do, slate devices are just too underpowered to realistically be useful for anything beyond simple analysis. In addition, I think the app model we use today won't go away anytime soon - HTML and JS have come a long way but just isn't capable of creating the high-end analysis experience we can get with real applications (particularly the kinds of apps we are talking about here). Funny enough, there was a recent study that concluded that people prefer dedicated iOS apps to HTML ones. I'm not shocked by that at all - I prefer the native apps too! They run faster, and tend to be easier to work with than web pages and single-page-apps. I think it validates what Microsoft has chosen to do with Windows 8 - we'll just have to see if developers forgive them for Silverlight ;-)

On the desktop side, in the Microsoft world, you really have three main choices for development today:

1) C++ (MFC, ATL, etc.). Gives you native, performant code but much harder to work in for most developers and certainly not ideal for people who aren't developers but are just trying to program to work in their real field of study. No .NET Bio here of course.

2) .NET and Windows Forms. .NET based so much easier to work with but is restricted to the older legacy Windows APIs.

3) .NET and WPF, also .NET based but provides a much more powerful rendering engine that is capable of higher-end visualizations than MFC can do (at least without dropping to really low-level rendering with DirectX). This also has the benefit of being "forward-compatible" as Windows 8 and WinRT both use the same basic technologies present here to build Windows Store applications.

For web based apps, I actually disagree - I think .NET is alive and well - there are a lot of systems based on ASP.NET MVC today and it's a thriving community. It generates straight HTML5 for the browser but uses C# (or your language of choice) on the server side so it could use .NET Bio to work with sequences/algorithms and send chunks of data to the client. My only problem with this is the there aren't a lot of great visualization libraries for Javascript today so you would end up pioneering a lot there. The desktop world has been around a lot longer and so it has more depth in that area.

It's hard to say where we'll go from here -- I do think that web apps are getting more powerful and perhaps someday we'll abandon the desktop world altogether and move to cloud based systems for everything - but I don't think that day has quite arrived yet.

-ms


On Thu, Mar 21, 2013 at 2:17 PM, evolvedmicrobe <notifications@codeplex.com> wrote:

From: evolvedmicrobe

Hi Mark,

Interesting thread. Given that silverlight is dead, do you know if that means that WPF is heading the way of the buffalo as well? The two technologies seemed to overlap quite a bit, and I can't imagine the death of one bodes well for the other.

I am sort of getting the sense that the trend to mobile is pushing good available options away from desktop development and towards technology that relies more on a server/client programming like html5/javascript. It seems that the server pushes data to client dynamic doesn't work well if using .NET since the browsers can't work with C#, which to me suggests that .NET would not be the platform to develop either the client or server side logic (though the other alternatives seem far more annoying than silverlight would have been).

I am far from an expert on recent developments in the software industry, but one of the things I love about c# is how easy it is to code and play with visualizations, and would be interested to know how other C# people are thinking about doing going forward for visualizations if it is being deprecated (or is it?)

-Nigel

Read the full discussion online.

To add a post to this discussion, reply to this email (bio@discussions.codeplex.com)

To start a new discussion for this project, email bio@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Mar 21, 2013 at 8:33 PM
It was quite descriptive and much helpful post. So much things you described on the behalf of other topic but it was also related with my subject in the end. This post really cleared my mind. Now I think I should use WPF with the combination of .NET bio library as my work demands enhanced UI with biological sequence analysis and in this way I will learn some new platform :)

Any suggestions, advice and tips for chart library available for WPF will be warmly appreciated :)
Coordinator
Mar 21, 2013 at 8:47 PM
There are several 3rd party commercial charting projects, but the open-source WPF toolkit has some built-in charting controls as well -- http://wpf.codeplex.com/releases/view/40535. In particular, check out these posts (some from the author of those components):




The charting toolkit was ported from Silverlight - so there are a lot of blog posts on the Silverlight version, but in general they apply to WPF as well. Also, note that the WPF toolkit was built for WPF 3.5, it was a proving ground for new controls and several of them were moved directly into WPF with 4.0 (at which point they stopped the project). Unfortunately the charting controls weren't considered sufficiently tested and didn't make the cut. However, they are usable in WPF 4 (and 4.5) without any problem, or you can take the source code and pull out just what you need as well.

mark


On Thu, Mar 21, 2013 at 3:33 PM, Saad_Qureshi <notifications@codeplex.com> wrote:

From: Saad_Qureshi

It was quite descriptive and much helpful post. So much things you described on the behalf of other topic but it was also related with my subject in the end. This post really cleared my mind. Now I think I should use WPF with the combination of .NET bio library as my work demands enhanced UI with biological sequence analysis and in this way I will learn some new platform :)

Any suggestions, advice and tips for chart library available for WPF will be warmly appreciated :)

Read the full discussion online.

To add a post to this discussion, reply to this email (bio@discussions.codeplex.com)

To start a new discussion for this project, email bio@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Mar 21, 2013 at 9:07 PM
A very warm thanks again! :)
Mar 21, 2013 at 9:54 PM
May I suggest you have a look of this http://seqcos.codeplex.com/

Given the source code is there, you can quickly come up your own program.

Best,

Dong
Mar 21, 2013 at 10:26 PM
great comments Mark and Nigel

like i tell in the last discussion http://bio.codeplex.com/discussions/437095, i think the future is in HTML5, javascript and CSS3, but, i think, that applications like pivot, implemented over silverlight are very usefull to comparative genomics and even to take another further step in bioinformatics through semantic web, applications like zentity and BL!P has a lot of potencial to infer new information, and i think is complicated migrate these tools to new technologies (is almost a silverlight HTML5 implementation), so what do you recommend? another technologies where we have to be focused is AZURE i think is something like BAAS (bioinformatics as a Service)

Markets, like the latinoamerican market (mi market), are not prepared yet, for bioinformatics mobile, also WP7 has a little piece of the market, i think responsive design is the answer.

regards from Colombia,
Mar 21, 2013 at 10:44 PM
xied75 wrote:
May I suggest you have a look of this http://seqcos.codeplex.com/

Given the source code is there, you can quickly come up your own program.

Best,

Dong
Thanks for informing about this. I have to play with this little bit to understand it.
Coordinator
Mar 21, 2013 at 11:43 PM
That's pretty nifty - thanks for sharing the link!

mark

On Thu, Mar 21, 2013 at 5:45 PM, Saad_Qureshi <notifications@codeplex.com> wrote:

From: Saad_Qureshi

xied75 wrote:
May I suggest you have a look of this http://seqcos.codeplex.com/

Given the source code is there, you can quickly come up your own program.

Best,

Dong
Thanks for informing about this. I have to play with this little bit to understand it.

Read the full discussion online.

To add a post to this discussion, reply to this email (bio@discussions.codeplex.com)

To start a new discussion for this project, email bio@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Mar 22, 2013 at 12:18 AM
Thanks Mark, long time no see!

@Saad, it is for you to learn WPF very quickly, not to learn what it does (i.e. checking sequence quality etc.) it is not relevent to Dot-Plot.

Also at least buy one book on WPF, it is not wise to learn WPF without a thick book, the learning curve is not smooth, XAML as one, and the Dependency properties stuff, if you are not used to XML and Declarative programming, it will take some time.

Last if you can put your work/code on codeplex or GitHub, then we might able to help you quicker when you got stuck, and we'll have a great Dot-plot tool in the end, all win situation.

Best,

Dong

p.s. remember to consider your usage spec/limitation, e.g. what is the biggest file you want the program to handle, the impact on I/O, loading speed, UI response time, and the chart control limits.
Mar 22, 2013 at 9:31 AM
Nice tips by the way I will try my best to learn WPF and I am quite convenient with HTML. I will buy the book as soon I can and i think codeplex will be more suitable as it is surrounded with .NET developer.
Coordinator
Mar 22, 2013 at 1:58 PM
Adam Nathan's WPF Unleashed is an excellent beginner's guide to WPF.

ms

On Fri, Mar 22, 2013 at 4:32 AM, Saad_Qureshi <notifications@codeplex.com> wrote:

From: Saad_Qureshi

Nice tips by the way I will try my best to learn WPF and I am quite convenient with HTML. I will buy the book as soon I can and i think codeplex will be more suitable as it is surrounded with .NET developer.

Read the full discussion online.

To add a post to this discussion, reply to this email (bio@discussions.codeplex.com)

To start a new discussion for this project, email bio@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Mar 22, 2013 at 3:25 PM
I have WPF 4 Unleashed by the same above author (Adam Nathan) which can be said as new edition of Windows Presentation Foundation Unleashed :)
Coordinator
Mar 23, 2013 at 10:59 AM
Wow, what a thread. Great post from mark covered the space very well indeed. At the desktop level WPF seems certain to be the best way forward for a little while yet, but there are a number of other possibilities - not least hooking up to charting in excel which we have done with the QUT.Bio work. We can share that code if needed. But the overwhelming message for Saad is to not reinvent the wheel. There is plenty of code out there to do this specific job or something closely related to it, and so you have a brilliant starting point.

jh
Mar 24, 2013 at 5:57 PM
Nice post jamesmhogan. I totally agree with you that's why I have started learning WPF. These post gave me a nice starting point for my project. Right now I am searching and testing nice chart library in WPF.
Coordinator
Mar 26, 2013 at 4:40 AM
Great to hear. Let me know how you get along
Developer
Mar 29, 2013 at 6:04 AM
Mark, thanks a lot for your thoughtful reply (And never apologize for posting a long quality response!). It is interesting to watch how the shift to mobile impacts the tools we use, but I completely agree with you that the desktop (hopefully) isn’t going away anytime soon. The comment about consuming versus generating content also definitely resonated with me. No one I know is diagnosing diseases or generating new science with some canned analysis that displays well with javascript, but I think a lot of scientific computing is slowed by the python/C++ choices made by many scientists who program.

For my two cents, in academia today I think performant code that is quickly written and lends itself to reuse is far less frequent than it should be, and I think java has enough short-comings where C# is the real winner in the realm the two inhabit. As an aside, I always found it interesting that despite the fact that computational biology today seems to be moving towards (or stagnating in?) C/C++/Python usage, the two most successful programs in the field, BEAST (phylogenetics +MCMC) and the GATK (for variant calling in sequence data) are both java programs, a language many people I know were only introduced to when they started working with one of these programs and became converts. They clearly owe a lot of their success to this language. However, they are also both java programs that accentuate how much worse java is than C# (e.g. the amount of awkward boxing and unboxing of primitives that happens in that code, the way linq could just make overly-complicated/boiler-plate code disappear, the importance of value type enums, the fact that Jython is no ironpython, it all adds up to a very clear difference).

Which is to say, I think the .NET Bio source could be the kernel of a deeply foundational code library, but it needs to grow a lot more. A friend of mine is the chief technology guy at a really well positioned biotech startup, and I was advocating that he move his codebase to C#. He had written a Scheme to .NET/CLR compiler for his undergraduate thesis way back when, liked the .NET framework and openly stated C# was better than java. However, he (understandably) went with java for reasons related to the inertia around the present computing environment.

I think you made a number of great suggestions relevant to structural changes in the code that would be an excellent idea to implement, so I would like to once more endorse them and offer what help I can for implementing them. Thanks again for the response!

Cheers,
Nigel

PS Saad, I actually started writing this (now long digression) to recommend a charting library. WPF/Silverlight are supposed to be able to take advantage of DirectX and other optimizations to render graphics faster. However, in my limited experience from moving a windows forms program to WPF in order to circumvent slow graphics rendering, I found the overhead introduced by all the dependencies in the visual element tree can easily offset any performance improvements one obtains from leaving the GDI. The Silverlight/WPF toolkit in particular seemed very slow for generating charts with many points. However, the visifire toolset started open source, is free, and seems to contain the core features of the expensive version (https://code.google.com/p/visifire/) . It might be worth a spin, (I use it now), but let me know what you settle on!
Mar 30, 2013 at 7:42 PM
Edited Mar 30, 2013 at 7:42 PM
Thanks for sharing your experience. I went to their site and found some very useful charts like "Visifire Point Chart". I will work on it and will try to use it for my work.
Apr 7, 2013 at 12:09 PM
Edited Apr 7, 2013 at 12:58 PM
Comming after almost one week, I have uploaded my project on codeplex as a Dot plot . I have used many logics of markjulmar (very thanks for your blogs about Dot plot). I am making my project on WPF and currently using VisiFire charting library. I am totally new about deploying project on code plex so if you feel or see any unregular things then you can advise me to do it in a rigth way. I want to know what type of liscence would be best at this stage when this project is underconstruction.
Coordinator
Apr 7, 2013 at 2:04 PM
Thanks for following through on this. as the project is not yet published, we can't yet see it. Do you want to share the project at this stage?
Apr 7, 2013 at 2:53 PM
Edited Apr 7, 2013 at 2:55 PM
There are four steps that I have to complete in order to publish that project and I have complete three of them. I am stuck at at liscening step. I am not sure what to choose. Actually I have uploaded my project if ever I get stuck or there is something I want to show, I can have my project online. Might be this place is not for these purpose because I am not fully aware.
Coordinator
Apr 9, 2013 at 4:49 PM
Saad, I have published our project on Codeplex but am by no means an expert. That said I would be happy to help you at least answer some questions about which license to choose. Do you have the options available and what questions do you have in that regard? As Jim noted we can't see your project in an unpublished state so it's hard to advise without knowing what specifically you are running into.

Rick
Apr 10, 2013 at 2:55 PM
Dear readers I have published my project and I set licence which is known as Common Development and Distribution License (CDDL). There are enough points to read but I want my project to be published so that I can ask question with my online code at my hand.
@Rickbe I have set above license and it will take atleast 3 or more days to understand the condition of the same license. Acutally I want anyone to use my project for any purpose. It would be a joke to say if anyone will use it for commercial purpose.
Coordinator
Apr 10, 2013 at 6:44 PM
Hi Saad,

Thanks for posting your project, it's great to see someone building apps with .NET Bio!

I would like to suggest that you make your compiled binary (or binaries) available for download from CodePlex, and also that you consider adding a few screenshots of your application to your page - currently someone visiting your page would have to download and compile your source code to see the application. I think you will find that a few pictures and an executable to download will make your work easier for others to use.

With best wishes,

Simon
Apr 11, 2013 at 4:50 PM
This is very interesting and much courageous post for me. I would like to thank you and others also :)
I will upload executable file along with its necessary libraries as soon as I can and pictures also.