Infer.NET

How to use posterior for testing in the LDA model

shakey — Sat, 21 Nov 2009 13:42:13 GMT

I want to test the lda model on new documents, I take the method on the baysian point machine example:

setting the phi in the test model to the posterior phi from the training model. However, I always get errors indicating violation of type constrain. Can anyone give me some help on this ? It seems SharedVariable is another kind of solution, but I cannot get it right either.

part of the code:

post_phi=engine.Infer[DistributionRefArray[Dirichlet,Vector]](phi)
test_post_phi=Variable[Vector].Random[DistributionRefArray[Dirichlet,Vector]](post_phi )

with (Variable.ForEach(test_CorpusSize)) as forBlock :

with (Variable.ForEach(test_DocSize)) as forBlock :

test_Z[test_CorpusSize][test_DocSize] = Variable.Discrete(test_theta[test_CorpusSize]).Attrib( ValueRange(TopicsNum))

with (Variable.Switch(test_Z[test_CorpusSize][test_DocSize])) as swBlock :

test_W[test_CorpusSize][test_DocSize] = Variable.Discrete(test_post_phi[ test_Z[test_CorpusSize][ test_DocSize] ])

error occurs when constructing a lda model with simple corpus

xgear — Sat, 21 Feb 2009 10:16:06 GMT

Normal 0 7.8 磅 0 2 false false false MicrosoftInternetExplorer4

setting of the model:

number of documents in corpus:12

number of topics:3

number of words(terms) in corpus:12

for simplicity,suppose that each document is composed of 2 words

Normal 0 7.8 磅 0 2 false false false MicrosoftInternetExplorer4

The whole corpus is show in the table below ,with each line representing a document.

Original corpus

After indexing ,the whole corpus is denoted as

university test

teacher student

teacher university

university student

economy bank

economy money

stock economy

money stock

goverment policy

goverment president

goverment military

president policy

0, 3

0, 1

0, 2

1, 2

4, 7

4, 5

4, 6

5, 6

8, 11

8, 9

8, 10

9, 10

after runing the program,the consle window show:

Compile model.....complilation failed.

then a "transform chain" window shows information "can only indexed by loop variables,not index0",it seems the position where the error occurs is near(in) "two using nest" of source code

By the way,can jagged array provide a array of a array,where the length of last array is not fixed,so I can remove the limit that each docoment is composed of 2 words.

Your help is appreciated!

Normal 0 7.8 磅 0 2 false false false MicrosoftInternetExplorer4

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

static void Main(string[] args)

{

int M = 12;//number of documents in corpus

int K = 3;//number of topics

int V = 12; //number of words(terms) in corpus

int Nm = 2;//suppose that each document is composed of 2 words

Range CorpusSize = new Range(M);

Range TopicsNum = new Range(K);

Range WordsNum = new Range(V);

Range DocSize = new Range(Nm);

double[] alpha={ 0.5, 0.5, 0.5 };

double[] beta = { 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1 };

VariableArray<Vector> theta = Variable.Array<Vector>(CorpusSize);

VariableArray<Vector> phi = Variable.Array<Vector>(TopicsNum);

theta[CorpusSize] = Variable.Dirichlet(alpha).ForEach(CorpusSize);

phi[TopicsNum] = Variable.Dirichlet(beta).ForEach(TopicsNum);

VariableArray2D<int> W = Variable.Array<int>(CorpusSize, DocSize);

VariableArray2D<int> Z = Variable.Array<int>(CorpusSize, DocSize);

using (Variable.ForEach(CorpusSize))

{

using (Variable.ForEach(DocSize))

{

Z[CorpusSize, DocSize] = Variable.Discrete(theta[CorpusSize]);

W[CorpusSize, DocSize] = Variable.Discrete(phi[Z[CorpusSize, DocSize]]);

}

W = Variable.Observed(new int[,] { { 0, 3 }, { 0, 1 }, { 0, 2 }, { 1, 2 }, { 4, 7 }, { 4, 5 }, { 4, 6 }, { 5, 6 }, { 8, 11 }, { 8, 9 }, { 8, 10 }, { 9, 10 } }, CorpusSize, DocSize);

InferenceEngine engine = new InferenceEngine();

Console.WriteLine(engine.Infer(Z));

}

A Question about "ConstrainEqualRandom"

Keri — Sat, 24 Oct 2009 02:09:44 GMT

Hi,

I am new to Infer.net, and I am trying to play with it.
As documented, there is a great way to attach constraint between
random variables using "ConstrainEqualRandom"...I made
a simple test showing below, but I cannot explain why Infer.net
gives me that result...

Could anyone explain how Infer.net generates such result? or I
used this function in a wrong way?..

What I want to test is suppose I have a prio distribution on A, and B.
I also know C = A & B... My intuition is if I have some constraint on
C, can I infer something about A,B,and C...

---------------------------------------------------------------------------------------------------------------------

       static void Main()
        {
            Variable<bool> PA = Variable.Bernoulli(0.3).Named("PA");
            Variable<bool> PB = Variable.Bernoulli(0.8).Named("PB");
            Variable<bool> PC = (PA & PB).Named("PC");

            Variable.ConstrainEqualRandom<bool, Bernoulli>(PC, new Bernoulli(0.6));

            InferenceEngine ie = new InferenceEngine();
            ie.ShowFactorGraph = true;

            Console.WriteLine("PA: " + ie.Infer(PA));
            Console.WriteLine("PB: " + ie.Infer(PB));
            Console.WriteLine("PC: " + ie.Infer(PC));
        }

----------------------------------------------------------------------------------------------------------------------

Compiling model...done.
Initialising...done.
Iterating:
.........|.........|.........|.........|.........| 50
PA: Bernoulli(0.375)
PB: Bernoulli(0.8214)
PC: Bernoulli(0.3214)
.

when I change Variable.ConstrainEqualRandom<bool, Bernoulli>(PC, new Bernoulli(0.6));
to some others like Variable.ConstrainEqualRandom<bool, Bernoulli>(PC, new Bernoulli(0.7));
Infer.net will give me different result as well.

Factor Graphs control

Valmir Meneses — Tue, 10 Nov 2009 19:24:15 GMT

Greetings,

I am using INFER.NET on my research (Bayesian Networks) and wish to control when/where to display the factor graph related to the inference engine currently used. Is there a method to save the factor graph without/instead of displaying it?

Thanks,

A question about LDA extention

msdy — Sat, 17 Oct 2009 05:23:12 GMT

Referring John's post for the LDA model, I am wonderring how to extend it to different data sets, similar to correspond LDA.

Suppose two data sets, both are discrete. Then we can use two LDAs to construct this model, as the figure.

I think most of the code are similar to John's post, but how I can index Z with X for G in infer .net? part of the our my code is as following, but I got errors. Can anyone help? Thank you.

var Z = Variable.Array(Variable.Array<int>(DocSize1), CorpusSize);
var W = Variable.Array(Variable.Array<int>(DocSize1), CorpusSize);

var G = variable.Array(Variable.Array<int>(DocSIze2), CorpusSize);

W.ObservedValue = docs1;

G.ObservedValue = docs2

    using (Variable.ForEach(CorpusSize))
    {
        using (Variable.ForEach(DocSize1))
        {
            Z[CorpusSize][DocSize1] = Variable.Discrete(theta[CorpusSize]).Attrib(new ValueRange(TopicsNum));
            using (Variable.Switch(Z[CorpusSize][DocSize1]))
            {
                W[CorpusSize][DocSize1] = Variable.Discrete(phi[Z[CorpusSize][DocSize1]]);
            }
        }

using (Variable.ForEach(DocSize2))
{

X[CorpusSize][DocSize2] = Variable.DiscreteUniform(docSizeVar[DocSize1]).Attrib(new ValueRange(DocSize1));

   using (Variable.Switch( X[CorpusSize][DocSize2]))
            {
                G[CorpusSize][DocSize1] = Variable.Discrete(phi[Z[CorpusSize][X[CorpusSize][DocSize2]]]);
            }
        }
    }

Binary Logistic Regression

rhasnani — Sat, 24 Oct 2009 18:06:07 GMT

Hello:

I am new to both logistic regression as well as Infer.NET. I am trying to build a binary logistic model and would appreciate feedback/comments/corrections on the below code. The model has two independent variables (and a third one is added to the vectors for constant coefficient). Based on the single observation example provided in the post 'Bayesian logistic regression' and some code posted in another post (I think it was multiple linear regression), this is what I have come up with and have serious doubts about whether it is done correctly:

Vector[] data = new Vector[] { new Vector(1.0, -3, 1), new Vector(2.0, -2.1, 1), new Vector(1.0, -1.3, 1), new Vector(2.0, 0.5, 1), new Vector(1.0, 1.2, 1 ), new Vector(1.0, 3.3, 1), new Vector(1.0, 4.4, 1), new Vector(1.0, 5.5,1 ) };
Range rows = new Range(data.Length);
VariableArray<Vector x = Variable.Constant(data, rows).Named("x");
Variable<Vector> w = Variable.VectorGaussianFromMeanAndPrecision(new Vector(new double[] { 0, 0, 0 }), PositiveDefiniteMatrix.Identity(3)).Named("w");
VariableArray<bool> y = Variable.Array<bool>(rows);
y[rows] = Variable.BernoulliFromLogOdds(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(x[rows], w), 1.0));
InferenceEngine engine = new InferenceEngine(new VariationalMessagePassing());
y.ObservedValue = new bool[] { true, false, true, false, false, true, false, true };
VectorGaussian postW = engine.Infer<VectorGaussian>(w);
txtResult.Text = "W = \n" + postW;

The output is:

W =
VectorGaussian(-0.3729 0.01771 0.2636, 0.2634   0.009538 -0.3017 )
                                       0.009538 0.01458 -0.02437
                                       -0.3017 -0.02437 0.4693

Please advise on whether the model is built correctly and how to read the output. I am assuming that b0 = 0.4693, b1 = -0.3017 and b2 = -0.02437.

All help is greatly appreciated.

Many thanks!

R Hasnani

A question about gaussian mixture models

euclid — Wed, 07 Oct 2009 11:37:55 GMT

Hi All,

I am completely new to both Infer and Gaussian Mixture Models (GMM). Infer.Net looks extensive, thanks for the effort on it. I am going to use it in a novel application.

At a glance, I thought that a GMM for a set of (random) observations could be obtained within the Infer.Net. I obtained these random observations from a system; but when I started going through the API, I felt quite lost.

Is there an easy starting point to obtain a GMM for random observations with unknown distribution?

Thanks!

Belief propagation with Infer.net

amchiclet — Sun, 18 Oct 2009 04:33:15 GMT

Hi,

I'm new to both belief propagation and Infer.net, so if my questions don't make sense, please point me in a good direction to understand the matter better which leads to asking better questions.

I have this algorithm on paper.

There are
1) a probability distribution function
2) a psi function
3) a phi function
4) a set of nodes structured as a tree
5) messages between the nodes all initialized to 1

From what I understand, we update the messages until they "saturate". This is called belief propagation, right?

I don't really know where to start modeling this structure in Infer.net because it seems like
1) the probability distribution function doesn't look like a standard one like beta, gaussian, etc. It uses products of phi's and psi's to some extent
2) I don't know how I would go about implementing the psi and phi functions
3) I'm not quite sure what variable I want to infer
4) the inference engine will use the belief propagation algorithm, and infer() will be called once. What if I need to see the message updates between each loop?

I've seen http://community.research.microsoft.com/forums/p/2434/3726.aspx#3726 and I think that an easy example like that will help me understand the best.

Thank you for your help.

MRF and Infer.NET

hr0nix — Sat, 17 Oct 2009 15:27:40 GMT

Can you please advice how one can model Markov random field in Infer.NET? I'd like to implement a simple image segmentation algorithm, but I'm really unsure about the way of specifying clique potentials in the model.

Thanks in advance.

SharedVariableArray

freddycct — Sun, 04 Oct 2009 04:50:50 GMT

Hi John,

Thanks for your reply

how do I convert this array here to a SharedVariableArray?

VariableArray<Vector> phi = Variable.Array<Vector>(topicsNumRange);

infer.net error on mono

hrotsos — Mon, 14 Sep 2009 10:37:54 GMT

I am currently trying to develop a simple bayesian classifier. I run approximately (generally I have a couple of function that break the inference process in small steps- db object is a simple interface to the database that return int[] on all functions mentioned here ) the following code:

Variable<Vector> modelClassDistribution;

VariableArray<Vector>[] modelParams;

        int tmp;
        // Infer.NET range for classes.
        Range modelClassRange = new Range(this.db.getClassCount());
        // Put a Dirichlet prior on the class distribution.
        arrClassDist = this.db.getClassDistribution(this.trainDataset);
        for (tmp = 0; tmp < arrClassDist.Length; tmp++)
          arrClassDist[tmp] += 1.0;
        modelClassDistribution = Variable.Dirichlet(modelClassRange, arrClassDist);

        for (int i = 0; i < this.INT_FIELDS.Length; i++) {
          this.modelParams[ i ] = Variable.Array<Vector>(modelClassRange);
          arrFeatDist = this.db.getScalarFeatureDistribution(this.trainDataset, i);
          dir = new Dirichlet[arrFeatDist.Length];
          for (tmp = 0; tmp < arrFeatDist.Length; tmp++) {
            for (j = 0; j < arrFeatDist[tmp].Length; j++)
              arrFeatDist[tmp][j] += 1.0;
            dir[tmp] = new Dirichlet(arrFeatDist[tmp]);
            prior = Variable.Constant(dir, modelClassRange);
          }
          using (Variable.ForEach(modelClassRange)) {
            this.modelParams[ i ][modelClassRange] = Variable.Random<Vector, Dirichlet>(prior[modelClassRange]);
          }
        }

int numTestpoints = 10;

      // Infer.NET range for the datapoints.
      Range testDataRange = new Range(numTestpoints);

      VariableArray<int> testClasses = Variable.Array<int>(testDataRange);
      VariableArray<int>[] testFeat = new VariableArray<int>[this.INT_FIELDS.Length];
      for (i = 0; i < this.INT_FIELDS.Length; i++) {
        testFeat[ i ] = Variable.Array<int>(testDataRange);
      }

      // Infer.NET variable array for the labels.
      using (Variable.ForEach(testDataRange)) {
        testClasses[testDataRange] = Variable.Discrete(modelClassDistribution);
      }

      for (i = 0; i < this.INT_FIELDS.Length; i++) {
        using (Variable.ForEach(testDataRange)) {
          using (Variable.Switch(testClasses[testDataRange])) {
            testFeat[ i ][testDataRange] = Variable.Discrete(this.modelParams[ i ][testClasses[testDataRange]]);
          }
        }
      }

        for (i = 0; i < this.field_count; i++) {
          testFeat[ i ].ObservedValue = this.db.getFeatureData(i);
        }

        dist = ie.Infer<Discrete[]>(testClasses);

And I get the following error:

Compiling model...
** (simpleBayes/bin/Debug/simpleBayes.exe:28682): WARNING **: Missing method System.Collections.Generic.ICollection`1.Contains in assembly /auto/homes/cr409/workspace/FlowClassifier/simpleBayes/simpleBayes/bin/Debug/Infer.Compiler.dll token a00007e

Unhandled Exception: System.TypeLoadException: A type load exception has occurred.
at #Bd.#QS.ConvertExpressionStatement (IBlockStatement bs, IExpressionStatement ies) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.DoConvertStatement (IBlockStatement bs, IStatement ist) [0x00000]
at #Bd.#QS.DoConvertStatement (IBlockStatement bs, IStatement ist) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertStatement (IBlockStatement bs, IStatement ist, System.Object key) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertBlock (IBlockStatement outputBlock, IBlockStatement inputBlock, System.Object key) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertMethodBody (IMethodDeclaration md, IMethodDeclaration imd) [0x00000]
at #Bd.#QS.ConvertMethodBody (IMethodDeclaration md, IMethodDeclaration imd) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertMethod (IMethodDeclaration md, IMethodDeclaration imd) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.DoConvertMethod (IMethodDeclaration imd, System.Object key) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertMethods (ITypeDeclaration td, ITypeDeclaration itd) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.ConvertType (System.Object owner, ITypeDeclaration td, ITypeDeclaration itd) [0x00000]
at MicrosoftResearch.Transforms.CopyTransform.Transform (ITypeDeclaration itd) [0x00000]
at MicrosoftResearch.Transforms.CodeTransformer.TransformToDeclaration (ITypeDeclaration typeDecl) [0x00000]
at MicrosoftResearch.Transforms.TransformerChain.TransformToDeclaration (ITypeDeclaration itd, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes) [0x00000]
at MicrosoftResearch.Infer.ModelCompiler.CompileWithoutParams (ITypeDeclaration itd, System.Reflection.MethodBase method, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes) [0x00000]
at MicrosoftResearch.Infer.Models.ModelBuilder.Compile (MicrosoftResearch.Infer.InferenceEngine engine, Boolean checkParamsAreSet) [0x00000]
at MicrosoftResearch.Infer.InferenceEngine.GetCompiledInferenceAlgorithm (IEnumerable`1 vars, Boolean checkParamsAreSet, Boolean inferOnlySpecifiedVars) [0x00000]
at MicrosoftResearch.Infer.InferenceEngine.InferAll (Boolean inferOnlySpecifiedVars, IEnumerable`1 vars) [0x00000]
at MicrosoftResearch.Infer.InferenceEngine.Infer (IVariable var) [0x00000]
at MicrosoftResearch.Infer.InferenceEngine.Infer[Object] (IVariable var) [0x00000]
at simpleBayes.Program.testModel () [0x00000]
at simpleBayes.Program.run () [0x00000]
at simpleBayes.Program.Main (System.String[] args) [0x00000]

Does this error comes from my code or is it an error on mono? The same code ruyns perfectly on Visual Studio, but I need to run this program on a linux platform.

dirichlet with non uniform prior

freddycct — Thu, 20 Aug 2009 09:39:49 GMT

Hi, I try to create an array of dirichlet distribution with non uniform priors but it wouldn't compile at all with these errors. Please help me. Thanks all.

Error 1 The best overloaded method match for 'MicrosoftResearch.Infer.Models.Variable.Dirichlet(double[])' has some invalid arguments C:\Documents and Settings\Administrator\My Documents\Visual Studio 2008\Projects\CPT\CPT\Program.cs 38 44 CPT

Error 2 Argument '1': cannot convert from 'MicrosoftResearch.Infer.Models.Variable<double[]>' to 'double[]' C:\Documents and Settings\Administrator\My Documents\Visual Studio 2008\Projects\CPT\CPT\Program.cs 38 64 CPT

static void Main(string[] args)

{

Range two = new Range(2).Named("two");

Range five = new Range(5).Named("five");

VariableArray2D<Vector> behaviour = Variable.Array<Vector>(two, five);

double[,][] alpha = new double[,][] { {

new double[]{ 1, 0, 0, 0, 0 },

new double[]{ 2, 1, 0, 0, 0 },

new double[]{ 3, 2, 1, 0, 0 },

new double[]{ 4, 3, 2, 1, 0 },

new double[]{ 5, 4, 3, 2, 1 }

{

new double[]{ 1, 2, 3, 4, 5 },

new double[]{ 0, 1, 2, 3, 4 },

new double[]{ 0, 0, 1, 2, 3 },

new double[]{ 0, 0, 0, 1, 2 },

new double[]{ 0, 0, 0, 0, 1 }

} };

VariableArray2D<double[]> alph = Variable.Constant<double[]>(alpha, two, five).Named("alph");

using (Variable.ForEach(two))

{

using (Variable.ForEach(five))

{

behaviour[two, five] = Variable.Dirichlet( alph[two, five] );

}

Console.WriteLine("Press any key to exit.");

System.Console.ReadKey();

}

Need help for a Bayesian Network Code

freddycct — Fri, 14 Aug 2009 06:25:16 GMT

Hi, I have the above Bayesian Network and I need some help for my code.

s has 5 discrete states, and follows the distribution defined by alpha

t has 2 discrete states, with beta prior

q has 5 discrete states with theta as dirichlet prior

static void Main(string[] args)

{

Range two = new Range(2);

Range five = new Range(5);

Variable<double> beta = Variable.Beta(1, 1);

Variable<bool> t = Variable.Bernoulli(beta);

double[] prior = { 1, 1, 1, 1, 1 };

VariableArray2D<Vector> alpha = Variable.Array<Vector>(two, five);

Variable<Vector> theta = Variable.Dirichlet(prior);

Variable<int> q = Variable.Discrete(five, theta);

using (Variable.ForEach(two))

{

using (Variable.ForEach(five))

{

alpha[two, five] = Variable.Dirichlet(prior);

}

//Breaking Symmetry

Vector denseBeta = new Vector(5, 10.0);

Dirichlet dirich = new Dirichlet(denseBeta);

Dirichlet[,] initAlpha = new Dirichlet[2,5];

for (int i = 0; i < 2; i++) {

for(int j=0;j<5;j++)

{

initAlpha[i,j] = new Dirichlet(dirich.Sample());

}

alpha.InitialiseTo(Distribution<Vector>.Array(initAlpha));

Variable<int> s;// = Variable.New<int>();

using (Variable.If(t))

{

using (Variable.Switch(q))

{

//s.SetTo(Variable.Discrete( alpha[Variable.Constant<int>(1),q] ));

s = Variable.Discrete(alpha[Variable.Constant<int>(1), q]);

}

using (Variable.IfNot(t))

{

using (Variable.Switch(q))

{

//s.SetTo(Variable.Discrete(alpha[Variable.Constant<int>(0), q]));

s = Variable.Discrete(alpha[Variable.Constant<int>(0), q]);

}

s.ObservedValue = 4;

InferenceEngine engine = new InferenceEngine();

//Console.WriteLine(engine.Infer(beta));

Console.WriteLine(engine.Infer(alpha));

Console.WriteLine(engine.Infer(theta));

Console.WriteLine(engine.Infer(t));

Console.WriteLine(engine.Infer(q));

Console.WriteLine("Press any key to exit.");

System.Console.ReadKey();

}

Are my codes correct? The parameters don't seem to change at all after inference

Re: Adding a new factor

julia — Thu, 13 Aug 2009 11:29:23 GMT

Sorry, I just got it. I was hoping to do it directly in the Factor class, but a new class has to be created. I had tried, but an invisible typo had made it bug. It all works now. My apology.

Adding a factor

julia — Thu, 13 Aug 2009 10:13:53 GMT

I am trying to add a factor to infer.net but it seems that I am missing something.

1) In Factor.cs I have added

public static double MyFactor (double a, double b, double c) { return (a-b)*(a+b-c) ; }

2) I created a file MyFactorVmpOp.cs containing

namespace MicrosoftResearch.Infer.Factors {

[ FactorMethod ( typeof(Factor), "MyFactor", typeof(double), typeof(double), typeof(double) ) ]

public static class MyFactorVmpOp {

public static Gaussian MyFactorAverageLogarithm( Gaussian A, Gaussian B, Gaussian C ){ };

public static Gaussian AAverageLogarithm( Gaussian MyFactor, Gaussian B, Gaussian C ){ };

public static Gaussian BAverageLogarithm( Gaussian MyFactor, Gaussian A, Gaussian C ){ };

public static Gaussian CAverageLogarithm( Gaussian MyFactor, Gaussian A, Gaussian B ){ };

}

3) In the file containing the main script, I have added the line

[assembly: MicrosoftResearch.Infer.Factors.HasMessageFunctions]

4) I call the factor using

Variable<double> x = Variable<double>.Factor(Factor.MyFactor, x,y,w);

but at compilation it says 'MicrosoftResearch.Factors.Factor' does not contain a definition for 'MyFactor'

It seems that it doesn't take into account the change in Factor.cs. I have read the help on that subject but there doesn't seem to be anything else to do... any idea?

Thanks, Julia

New release: Infer.NET 2.3 beta 1

jwinn — Wed, 12 Aug 2009 23:07:42 GMT

The latest beta release includes new factors (Logistic, Binomial), more efficient inference, improved support for F#, IronPython and Mono, partial support for parallel inference and Gibbs sampling and more.

See this blog post for details and, as ever, post any questions you have here on the forum!

The Infer.NET team

problem with Variable.Array

julia — Wed, 29 Jul 2009 21:06:31 GMT

I am struggling with these four lines of code, I don't understand why I am getting an error.

******************************************************

VariableArray<VectorGaussian> D = Variable.Array<VectorGaussian>(classRange);
D[classRange] = Variable.New<VectorGaussian>().ForEach(classRange);

VariableArray<Vector> x = Variable.Array<Vector>(classRange).Named("x");
x[classRange] = Variable.Random<Vector, VectorGaussian>(D[classRange]);

gives me an error at the execution of line 2:
Unhandled Exception: System.InvalidOperationException: vVectorGaussian[ ]1[classRange] was not defined in this condition block.

******************************************************

I'm guessing the problem comes from the fact that D is not properly initialized... but how can I define it other than what I have done?

Is there another way to store an array of distributions? My intention was to update the "observed value" of the distribution array D with the results of the inference, using
D.ObservedValue = Distribution.ToArray<VectorGaussian[ ]>( (IDistribution<Vector[ ]>) inferred[0])

If I store D as an array of VectorGaussian instead of a VariableArray, then I am in trouble for the indexing, as x must be indexed by a range and D by an integer...

Any help welcome! Many thanks,

Julia

Infer.NET: Bayesian logistic regression

Ferrum — Thu, 14 May 2009 14:23:54 GMT

Does Infer.NET have factors for Bayesian logistic regression? If there are some in-house factors for VMP, do they use Tommi Jaakkola's bound with auxiliary variational parameters?

Specifying large Conditional Probability Tables of discrete events

freddycct — Tue, 11 Aug 2009 14:41:42 GMT

Suppose I have 3 random variables x, y and z. Each random variable can take 10 possible states. and let

x depend on y and z, hence, I need to specify the conditional probability table of P(x | y, z )

If I specify the conditional probabilities using the "using Variable.if(case) .... " statements, then I need to type alot of statements.

I understand there is a ifblock to use but there's no simple example to illustrate how to use it.

Is there any help on this or a simple work around?

Probability Density Function, Integration, Area under Curve, Continuous Random Variable

freddycct — Tue, 11 Aug 2009 05:44:53 GMT

Hi,

So I define a Beta Distribution in this way.

Variable<double> f = Variable.Beta(a, b);

which mathematically means

p(f) = ((a+b - 1) ! / (a-1)!) * (b-1)! * f ^(a-1) * (1-f)^(b-1)

My question is, how do i obtain the value for P( c < f < d ) ?

What I really need is an integration function of the beta density function.

Thanks for reading.

How do I create SharedVariable Arrays for Dirichlet distributed r.v.s?

laura — Tue, 17 Mar 2009 16:15:00 GMT

I am trying to do inference in batches using probabilistic submodels with shared variables.

I implemented an LDA style topic model. Now I want to process batches of documents to learn topic-characteristic word-distributions (p(word | topic) a.k.a. phi) that explain the whole document collection well. phi is a multinomial parameter drawn from a symmetric Dirichlet distribution we assume as given.

Since the number of topics is unknown during model compilation, I tried to create an array of shared variables (to be precise a SharedVariableArray) for maintaining the messages of phi.

I am stuck in constructing this SharedVariable array...

The documentation says

public static SharedVariableArray<DomainType> Random<DistributionArrayType>(
	Range range,
	DistributionArrayType prior
)

I assume that in my case the DomainType is Vector, and the DistributionType is probably a DistributionRefArray with T Dirichlet. 
The following code compiles

    let prior = new DistributionRefArray<Dirichlet,Vector>((RandomDirichletArray cRange.SizeAsInt vNum):Dirichlet[])
    let phiShared = SharedVariable<Vector>.Random<DistributionRefArray<Dirichlet,Vector>>(cRange,prior )
    let phi = phiShared.GetCopyFor(model)


But I get an error message complaining

MessageTransform failed with 4 error(s) and 0 warning(s):
System.Double is not an array type.
  System.Double is not an array type. in
Vector sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange__marginal
System.Double is not an array type.
  System.Double is not an array type. in
Vector sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange_
  Argument type could not be found for argument 'Def' in
sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange__uses = Factor.ReplicateWithMarginal<Vector>(sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange_, sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange__marginal)
  Argument type could not be found for argument 'enterOne' in
sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_cRange_cond_f_uRange__sRange_ = Gate.EnterOne<Vector>(f_cases_rep0_uses[0][cRange], sharedVector__DistributionRefArrayLtDirichlet_VectorGt0_rep0_rep0[sRange][cRange], cRange)


Any ideas?

Laura

multinomial distribution?

Junming Huang — Tue, 03 Mar 2009 10:13:29 GMT

Hi, I'm a beginner of Infer.NET. I've got a problem when trying to model a multinomial distribution with Infer.NET: I cannot find multinomial distribution neither in Infer.Distributions nor in any documents. Is multinomial distribution not a built-in one? If not, it seems I have to manully realize it in code.

Thanks very much.

Discrete point masses with point 4 and 3 dimensions

laura — Tue, 03 Mar 2009 18:58:17 GMT

In case you add some checks for ISO/OSI layer 8 errors (aka user-error):

I just figured (by accident) that I can create dirichlet point masses with three dimensions, where the point is set to 4.

Laura

Stuck with "Replace Expression: #i.#Qb" or "Can only index by loop variables"

laura — Fri, 06 Feb 2009 14:30:32 GMT

Trying to follow Tom's hints in the thread on "Work around plates by manual variable synchronization", I got stuck in a "Not Implemented Exception: Unhandled expression type in ReplaceExpression ( ): #i.#Qb". If factor graph and transformbrowser is enabled, this error message occurs between the two, but I have no idea what it is trying to tell me...

The model maintains one array of edgeLabel variables, indexed by eRange. Each edgeLabel is drawn from a Discrete distribution.

Each user has an array over his friends. From this I creates a Constant Array called edgeIndexData which points to the edgeIndex of that friendship. I.e. if the edge between user 0 and his second friend has index 5, then edgeIndexData.[0].[2]=5.

For each user u do

- draw a currentFriend ~ some distribution

- currentEdge = edgeIndex.[ u].[f] // deterministic, given f

- currentLabel = edgeLabel2.[currentEdge]

- do something with the label

I assume that the problem is related to using the right switches. The following code wraps currentFriend, currentEdge and currentLabel in a switch, and gets the exception.

                f.[uRange].[sRange] <- Variable.Discrete(psi.[uRange]).Attrib(new ValueRange(fVaryRange))

                let currentFriend = f.[uRange].[sRange]
                using (Variable.Switch(currentFriend)) (fun _ ->
                    let currentEdge = (edgeIndexData.[uRange].[currentFriend]).Attrib(new ValueRange(eRange))
                    using (Variable.Switch(currentEdge)) (fun _ ->
                        let currentLabel = ((edgeLabel2.[currentEdge])).Attrib(new ValueRange(cRange))
                        using (Variable.Switch(currentLabel)) (fun _ ->
                            // do something with currentLabel
                            )
                        )
                    )
               )

Then I was thinking that the Switch around currentEdge is sort of unnecessary, since the relation between currentFriend and currentEdge is deterministic. Omitting this switch I get the following error.

ChannelTransform failed with 2 error(s) and 0 warning(s):
Can only index by loop variables, not 'uRange'. in
vint____5[uRange] = Factor.GetItems<int>(edgeLabel2_rep0[uRange], edgeIndexData[uRange])
Can only index by loop variables, not 'uRange'. in
vint____5_cases = Gate.CasesInt(vint____5[uRange][fVaryRange])

I checked that uRange and fVaryRange are outer loops of these statements.

vint____5 could not be named, but it containes the edgelabels of a user's friends.

            for(int uRange = 0; uRange<vint0; uRange++)
            {
                vint____5[uRange] = Factor.GetItems<int>(edgeLabel2, edgeIndexData[uRange]);
            }

Any ideas?

Laura

Bayesian statistic theory question

freddycct — Fri, 31 Jul 2009 06:59:31 GMT

Hi, I am a beginner in the Bayesian approach of machine learning. I have read some literature on Bayesian Inference and parameter learning. I have a few queries that I hope to find an answer to.

In the case of coin toss, we often use a Beta Distribution as a conjugate prior for inferring the probability of the next coin toss landing on heads assuming that we have observed a series of coin tosses. I understand that Beta Distribution has a convenient property where the posterior is also a Beta Distribution.

1) My question is how did the Beta Distribution come about? How do we derive the Beta Distribution? I know that wikipedia has a derivation of Dirichlet Distribution using Gamma Distribution. But that is not intuitive and fundamental enough.

2) Can we use a Binomial Distribution as a prior for bayesian inference and prediction?

3) Finally, I read that discrete random variables should not have continuous random variables as parents. But why do we use Beta Distribution which is a continuous function as parent of a discrete event?

Pardon me if these questions sound silly. But I really hope to find an answer.