SP8-Choice of modeling distribution for frequency

[Nishi]

Hi
I am studying for SP8 IAI examination.

The recent IFoA course material under chapter-14-Rating using F/S and B/C approaches-Choice of distributions (Page 18) states that:

"For frequency, a negative Binomial distribution is commonly used in practice, because it allows for dependencies between claims, whereas the Poisson distribution assumes successive claims are independent of each other. "

I understand the independence condition in Poisson distribution. However I am unable to understand how does Negative Binomial distribution allow for dependencies between claims?

The old (2013) CT3 course notes start the description for Geometric Distribution by considering independent, identical Bernoulli trials. And negative Binomial is generalization of Geometric distribution. So where does the dependence part come in?

Also, the Negative Binomial distribution models number of trials or failures before kth success. How does this work when we are modeling frequency to determine premiums? For example, What will be trials or failures when I am trying to determine number of motor accident claims using NB distribution?

 

[Ian Senator]

I'm no stats expert, but I think we might be taking the 'dependency' issue to literally. Negative Binomial distributions are useful in that the variance can be much greater than the mean, so if there is significant uncertainty, it can be a more appropriate distribution than the Poisson distribution, which simply has mean=variance.
Subject CS2 now covered the stats behind these distributions, but I don't think that's going to help here.
Also worth noting that this subject is covered in a bit more detail in a 'GRIP' report (an actuarial paper on pricing published many years ago), but I think that's going beyond the syllabus.
Also see Chapter 15, page 36 for another reference to the suitability of the NB distribution when modelling aggregate claims.
We'll ask the IFoA to perhaps consider expanding on this part of the Core Reading in future years, although it might be getting off track as it may well be too statistical for SP level exams.

 

[Nishi]

Hi Ian

Thanks for the reply. I understand the difference in mean and variance for different distributions.

I understand that this sort of question might not be asked in the SP level exams. But the technicalities would be important when choosing different distributions in the modeling process so was trying to understand the dependence part.

Also, could you please clarify the part related to how would we define number of trials/failure when modeling frequency using Negative Binomial distribution?

 

[CapitalActuary]

If claims occur according to a Poisson process, then the independent increments property applies. That is, the number of claims occurring between time 0 and time 1 is independent from the number between times 1 and 2 (and for any other non-overlapping time intervals).

The Poisson distribution just describes the count of a Poisson process over a specified time window.

Recall a Poisson process has exponential wait times between events. It follows from the memoryless property of the exponential distribution then that wait times between events are independent. Hence a Poisson distribution, which describes the number of events over a specific time window, assumes independent times between events.

The extra parameter on the NB distribution just allows you to get a better fit to data.

Two things which could stop a distribution being Poisson are: 1) clustering and 2) a mixture distribution, as you mentioned.

1) Say we have events which happen independently, but each one can give rise to more than one claim. Then the count of events is Poisson but the count of claims wouldn’t be Poisson. The extra parameter on the negative binomial will allow us to get a better fit to this data than Poisson allows, while still being a ‘Poisson-like’ count. In this case there are still independent increments, as you pointed out - but it wouldn’t be Poisson because more than one claim can happen at once which is not a feature of a Poisson process.

2) Say we are counting the number of claims arising from policyholders. Each policyholder we might model as having a Poisson number of claims, but the rate of claims might be different for each policyholder. If we assume the rate is gamma distributed then we’d fit a negative binomial distribution (see the Gamma-Poisson mixture section of the negative binomial distribution Wikipedia page). But, even if the rate isn’t gamma distributed we’d get a better fit from the NB than Poisson simply because we have the extra parameter.

The above examples aren’t fully rigorous, but hopefully it gives some background. I think the main point is to give an idea of when you might want to fit a Poisson vs a NB distribution. If the events/claims/whatever you’re counting happen independently without clustering then a Poisson should provide a reasonable fit. Otherwise negative binomial would be the usual choice.

If you’re fitting a NB distribution you generally don’t think of it as fitting a number of trials parameter and a probability parameter. Instead you fit mean and variance (or mean and CoV) to your data. This is a pretty common thing to do, eg for lognormal distributions actuaries will rarely look to fit mu and sigma but instead just fit the mean and the CoV. You can work out the parameters algebraically. Hope this makes sense.

So, you have historical claims frequencies and you find the mean and variance of this data. Then you do some algebra to find whatever parameters you need for the NB distribution, since you know algebraic expressions for the mean and variance in terms of the parameters. (Noting that if mean=variance you should just fit a Poisson anyway.)

 

[Nishi]

If claims occur according to a Poisson process, then the independent increments property applies. That is, the number of claims occurring between time 0 and time 1 is independent from the number between times 1 and 2 (and for any other non-overlapping time intervals).

The Poisson distribution just describes the count of a Poisson process over a specified time window.

Recall a Poisson process has exponential wait times between events. It follows from the memoryless property of the exponential distribution then that wait times between events are independent. Hence a Poisson distribution, which describes the number of events over a specific time window, assumes independent times between events.

The extra parameter on the NB distribution just allows you to get a better fit to data.

Two things which could stop a distribution being Poisson are: 1) clustering and 2) a mixture distribution, as you mentioned.

1) Say we have events which happen independently, but each one can give rise to more than one claim. Then the count of events is Poisson but the count of claims wouldn’t be Poisson. The extra parameter on the negative binomial will allow us to get a better fit to this data than Poisson allows, while still being a ‘Poisson-like’ count. In this case there are still independent increments, as you pointed out - but it wouldn’t be Poisson because more than one claim can happen at once which is not a feature of a Poisson process.

2) Say we are counting the number of claims arising from policyholders. Each policyholder we might model as having a Poisson number of claims, but the rate of claims might be different for each policyholder. If we assume the rate is gamma distributed then we’d fit a negative binomial distribution (see the Gamma-Poisson mixture section of the negative binomial distribution Wikipedia page). But, even if the rate isn’t gamma distributed we’d get a better fit from the NB than Poisson simply because we have the extra parameter.

The above examples aren’t fully rigorous, but hopefully it gives some background. I think the main point is to give an idea of when you might want to fit a Poisson vs a NB distribution. If the events/claims/whatever you’re counting happen independently without clustering then a Poisson should provide a reasonable fit. Otherwise negative binomial would be the usual choice.

If you’re fitting a NB distribution you generally don’t think of it as fitting a number of trials parameter and a probability parameter. Instead you fit mean and variance (or mean and CoV) to your data. This is a pretty common thing to do, eg for lognormal distributions actuaries will rarely look to fit mu and sigma but instead just fit the mean and the CoV. You can work out the parameters algebraically. Hope this makes sense.

So, you have historical claims frequencies and you find the mean and variance of this data. Then you do some algebra to find whatever parameters you need for the NB distribution, since you know algebraic expressions for the mean and variance in terms of the parameters. (Noting that if mean=variance you should just fit a Poisson anyway.)

Thank you so much for the post. It is indeed insightful and clarifies my question and doubt.
Thanks again!