markov | PHP Markov chain text generator | Natural Language Processing library

When running the reproducible code at the bottom, I get an error message when I try transposing the reactive data table. In the image below I show the error message when attempting to transpose (via click of the radio button), with an overlay of my comments.

I know the problem lies in the re-dimensioning of the data table when running the t() (transpose) function in the code (the line underneath the only # in the code in the server section, starting with data = if(input$transposeDT==...). I have tried many iterations of results()[ , ] in the t() function with no luck yet. Please, does anyone have any guidance here?

I am working on a research project and a while back I asked this question on Mathematics Stack Exchange, where I was looking for a way to calculate the variance of the period-to-period change in income given a transition matrix, where each state corresponds to a log level of income in a vector, which is given. I want to calculate what the variance of an individual's change in income is over some n number of periods given that they began in each state. My state space consists of 11 states, so I hope to end up with a vector consisting of 11 different variances. When I asked the question, I received a satisfactory answer, but I am running into some issues when trying to code it in R I was hoping to receive help with.

I have created this piece of code to calculate the variances:

I am trying to generate a Markov simulation using a specific sequence as start, using the mchmm library coded with scipy and numpy. I am not sure if I am using it correctly, since the library also has Viterbi and Baum-Welch algorithms in the context of Markov, which I am not familiar with.

To illustrate, I will continue with an example.

I am trying to reconstruct a Markov process from Shannons paper "A mathematical theory of communication". My question concerns figure 3 on page 8 and a corresponding sequence (message) from that Markov chain from page 5 section (B). I just wanted to check if I coded the right Markov chain to this figure from Shannons paper:

Here is my attempt:

I am building a transition matrix for some markov chains, so I need all rows to add up to 1.

I create a random ten-state 10 x 10 matrix with:

I am approaching a problem that Keras must offer an excellent solution for, but I am having problems developing an approach (because I am such a neophyte concerning anything for deep learning). I have sales data. It contains 11106 distinct customers, each with its time series of purchases, of varying length (anyway from 1 to 15 periods).

I want to develop a single model to predict each customer's purchase amount for the next period. I like the idea of an LSTM, but clearly, I cannot make one for each customer; even if I tried, there would not be enough data for an LSTM in any case---the longest individual time series only has 15 periods.

I have used types of Markov chains, clustering, and regression in the past to model this kind of data. I am asking the question here, though, about what type of model in Keras is suited to this type of prediction. A complication is that all customers can be clustered by their overall patterns. Some belong together based on similarity; others do not; e.g., some customers spend with patterns like $100-$100-$100, others like $100-$100-$1000-$10000, and so on.

Can anyone point me to a type of sequential model supported by Keras that might handle this well? Thank you.

I am trying to achieve this in R. Haven't been able to build a model that gives me more than about .3 accuracy.

I am trying to write a small program for Markov Decision Process (inventory problem) using Python. I cannot figure out why the program outputs two identical matrices (for profit and decision matrices). The programming itself has some problems too because the last two columns are all zeros, which should not happen. Any help on the program itself would also be super helpful!

I am visiting a restaurant that has a menu with N dishes. Every time that I visit the restaurant I pick one dish at random. I am thinking, what is the average time until I taste all the N dishes in the restaurant?

I think that the number of dishes that I have tasted after n visits in the restaurant is a Markov chain with transition probabilities:

I need help with understanding the shaping theorem for MDPs. Here's the relevant paper: https://people.eecs.berkeley.edu/~pabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf it basically says that a markov decision process that has some reward function on transitions between states and actions R(s, a, s') has the same optimal policy as a different markov decision process with it's reward defined as R'(s, a, s') = R(s, a, s') + gamma*f(s') - f(s), where gamma is the time-discount-rate.

I understand the proof, but it seems like a trivial case where it breaks down is when R(s, a, s') = 0 for all states and actions, and the agent is faced with the path A -> s -> B versus A -> r -> t -> B. With the original markov process we get an EV of 0 for both paths, so both paths are optimal. But with the potential added to each transition we get, gamma^2*f(B)-f(A) for the first path, and gamma^3*f(B) - f(A) for the second. So if gamma < 1, and 0 < f(B), f(A), then the second path is no longer optimal.

Am I misunderstanding the theorem, or am I making some other mistake?