Hi everybody,
could anyone provide an simple example how to build a Bayesian network with Infer.NET, e.g. chest clinic?
Regards
P. Matthis
Below is the C# code for the classic wet grass/sprinkler/rain Bayesian network, as found in Kevin Murphy's tutorial, amongst other places. Notice that the conditional probability tables (CPTs) are represented by if blocks or nested if blocks, rather than in tabular form. In general, this allows any structure in the CPTs to be explicitly represented, which can often be exploited to make inference faster and/or more accurate.
To perform parameter learning in a discrete Bayesian network of this form, create a double random variable with a Beta prior and use it instead of the constants in any of the Variable.Bernoulli() calls.
Variable<bool> cloudy = Variable.Bernoulli(0.5);
The first part of the code defines the Bayesian network. The second part applies the default algorithm, expectation propagation, to the model, which in this case is equivalent to loopy belief propagation. We compute the posterior marginals for rain and sprinkler given that the grass is wet and then given that both the grass is wet and it is not cloudy.
Note that EP is an approximate algorithm in graphs with loops, like this one. Infer.NET is designed for approximate inference in large graphs - there are already many packages that can apply junction tree methods to get exact inference in small graphs. In this case, the first two answers are in a loopy graph and the second two are in a non-loopy graph (the observation on cloudy breaks the loop) and so the second two answers are exact whilst the first are approximate. The code prints out:
Best, John
Thank you, John!
I am working in a bank project and we try to apply Bayesian networks to portfolio risk analyses and estimations, we have rather large networks with flat structure (only few loops). We would like to read in networks from databases or flat files and look for sound tools for inference and simulation. Are there methods in Infer.Net to read networks, e.g. in HUGIN/ or dne-format? Intend you to implement importance sampling?
Best Regards
Peter
Hi Peter,
In designing Infer.NET we decided to focus on the inference engine itself rather than on I/O or on making a GUI. It should be straightforward to write code to read in a database or file in whatever format you are using and construct a model using the Infer.NET modelling API.
We are looking to implement some form of sampling in a future release - it is most likely that we will start with Gibbs sampling. However, you may be able to use Infer.NET as a subroutine in an importance sampler e.g. using the EP posterior as the proposal distribution. Bear in mind that importance sampling does not work well unless the proposal distribution is close to the true distribution - and this becomes increasingly important as the dimensionality of your problem increases (see section 29.2 of Information Theory, Inference and Learning Algorithms by David MacKay for a discussion of the weaknesses of importance sampling).
For those of us who aren't familiar with c# and .net., and do not have visual studio, this familiar example would be a good one to use for an excruciatingly detailed walk-through of building and using a model. What *exactly* goes into the source file (including imports, etc.)? How do I compile? How do I run? I can't find quite enough detail in the tutorials to get started from my current (low) level.
I've had some success with VIBES, so I have high hopes for infer.net--thanks for making this!
Sure, I can provide some more detail. When you installed Infer.NET, it created a folder "My Documents\Infer.NET 2.2". In this folder you can find "bin\Debug" as well as "Example Solutions\SimpleGaussian\Program.cs". This file has the complete set of imports and definitions you need. You can delete the entire contents of Main and replace with the code John sent above. To compile, you need to link Program.cs with two Infer.NET dlls which are located in the "bin\Debug" folder mentioned above. The dlls are Infer.Compiler.dll and Infer.Runtime.dll. This is done automatically by Visual Studio when the SimpleGaussian.sln solution file is opened.
And note that, if you don't have Visual Studio, you can download the C# Express Edition for free.
I hope other c#/.net beginners might benefit from a summary of my adventures so far...
If you want to compile at the command line:1. You need to add the directory where csc.exe (c# compiler) lives to your path. Mine was in "C:\WINDOWS\Microsoft.NET\Framework\v3.5".2. To compile, you have to link to "Infer.Compiler" and "Infer.Runtime", which are both in "infer.NET 2.2\bin\Debug" (thanks, Tom).3. The obvious compile fails:>csc program.cs /r:..\..\bin\Debug\Infer.Compiler.dll /r:..\..\bin\Debug\Infer.Runtime.dllActually, compiling finishes without protest, but the executable fails with a "System.IO.FileNotFoundException" re. Infer.Compiler. Is there some environment variable or read permission I have to fix?4. As a temporary fix, until a c# expert is handy, I moved the source file to the bin\Debug\ directory, and compiled it there. The executable then ran just fine.To use Visual C# 2008 Express Edition (thanks, John), this sequence seems to work: 1. Make sure you have a sandwich and a good book before starting VS install.2. Once you've got VS open, make/name a new project ("File -> New Project..."). 3. Go to "Project -> Add Existing Item..." to add an existing source file (e.g., program.cs).4. Go to "Project -> Add Reference..." to link the files "Infer.Compiler" and "Infer.Runtime" in "bin\Debug".5. Go to "Build -> Build Solution" to compile.6. Go to "Debug -> Start Without Debugging" to double-check that it runs.7. Save the project. The executable is named after the project, not the source file, and lives a couple of directories deep in the project's tree, in \bin\Release.
Hi all,
I need to extend my Bayesian Network (which is similar to this Sprinkler example) to include training data, how can I load data using Infer.Net? A simple example would be most helpful.
Thanks,xyc
Infer.NET does not provide built-in routines for loading data. Because it is called from within a .NET language (e.g. C#) you just load the data using .NET routines of your own and attach it to the model by setting the ObservedValue of the corresponding variable. Note that .NET has a lot of support for reading data from files or from databases. For example if you have a VariableArray<double> in your model called x, then you load in an ordinary .NET double array using a custom C# routine and attach it to the model using x.ObservedValue = MyDoubleArray.
Best,
John W.
Thanks for the reply John.
On another note, I'm trying to model the following network - worked example 3 from the link: http://controls.engin.umich.edu/wiki/index.php/Bayesian_network_theory#Worked_out_Example_3.Is there a clean way in Infer.NET to answer the question:What is the probability that a person will develop a blood clot as a result of both medication and trauma, and then have no medical implications?The answer is of course: P(N, BC, M, T) = P(N | BC) P(BC | M, T) P(M) P(T) = (0.25) (0.95) (0.2) (0.05) = 0.2375%, but I have been unable to generate this chain without explicitlyspecifying the conditionals directly. Is there some clean way of infering P(N,BC,M,T) as listed?The code below shows the network setup and a related run to infer bloodClot and Nothing when the medication and trauma are observed.Thanks,xyc
public void Run() {
Variable<bool> medication = Variable.Bernoulli(0.2).Named("medication");
Variable<bool> trauma = Variable.Bernoulli(0.05).Named("trauma");
Variable<bool> bloodClot = Variable.New<bool>().Named("bloodClot");
Variable<bool> heartAttack = Variable.New<bool>().Named("heartAttack");
Variable<bool> nothing = Variable.New<bool>().Named("nothing");
Variable<bool> stroke = Variable.New<bool>().Named("stroke");
using (Variable.If(medication)) {
using (Variable.If(trauma)) {
bloodClot.SetTo(Variable.Bernoulli(0.95).Named("b|m,t"));
}
using (Variable.IfNot(trauma)) {
bloodClot.SetTo(Variable.Bernoulli(0.3).Named("b|m,~t"));
using (Variable.IfNot(medication)) {
bloodClot.SetTo(Variable.Bernoulli(0.6).Named("b|~m,t"));
bloodClot.SetTo(Variable.Bernoulli(0.9).Named("b|~m,~t"));
using (Variable.If(bloodClot)) {
heartAttack.SetTo(Variable.Bernoulli(0.4).Named("h|b"));
nothing.SetTo(Variable.Bernoulli(0.25).Named("n|b"));
stroke.SetTo(Variable.Bernoulli(0.35).Named("s|b"));
using (Variable.IfNot(bloodClot)) {
heartAttack.SetTo(Variable.Bernoulli(0.15).Named("h|~b"));
nothing.SetTo(Variable.Bernoulli(0.75).Named("n|~b"));
stroke.SetTo(Variable.Bernoulli(0.1).Named("s|~b"));
// Inference
InferenceEngine ie = new InferenceEngine(new VariationalMessagePassing());
trauma.ObservedValue = true;
medication.ObservedValue = true;
Console.WriteLine("nothing and bloodClot: " + ie.Infer(nothing & bloodClot));
It sounds like you are trying to compute the probability of the joint event (trauma=true, medication=true, nothing=true, bloodClot=true). In Bayesian network terminology, this is the problem of computing the probability of observed data or "evidence". To compute this in Infer.NET, you can use the method described at: http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Computing%20model%20evidence%20for%20model%20selection.aspx. Basically you wrap your model and observations in an If block, and query the variable controlling the If block.