How to run a program in WCF?

Question

I am new to WCF and i am designing a project in which i want to run a crawler program (coded in c#) which crawlers some websites and it stores the crawled data in the tables of database (sql server db). I want that crawler runs repeatedly after 30 minutes and updated the database.

I want to then use the service on my hosted platform so that i can use the data from tables in web form (i.e. .aspx page)

Is it okay to use WCF for this purpose ?

Please suggest me how to move on ?

Thanks

Answer 1

You could use a WCF service to do this but I would go for another setup:

1. I'd build a Windows application that is scheduled to run every 30 minutes by the Windows Task Scheduler. A simple console application might be fine.

2. I'd use a Web application (possibly ASP MVC) to query the database.

As you can see there is no need to use WCF at all.

An exception can/must be made when the server is not yours but you are using a hosting provider who doesn't allow you to schedule a Windows task. In that case you might want to run the crawling process by hand through the web application and have it repeat itself after 30 minutes.

Some hosting providers do allow the scheduling of tasks but in a different way so it might be worth to investigate.

Answer 2

Windows Communication Foundation (WCF) is responsible for communication between 2 points with different channel technology. you will use WCF if you want to send/receive some data between two point regardless channel technology (TCP/UDP/NetPipe/MSMQ , ...)

But you first need to design you crawler application which is configured to fetch data from your target web sites, then you need to design a schedular application using http://quartznet.sourceforge.net/
to run your crawlers.

after running and storing your web pages you can use WCF if you need to do replication or synchronization with center server but it is optional