Basic-python | some knowledge about basic python | Graph Database library

Several places online, including answers in Stack Overflow (such as this, this & this), mention that Python iterators must implement both the __next__ method (which I understand) and the __iter__ method. These places rightfully conclude that all iterators are also iterables. Even PyCharm issues a type warning if a variable which is annotated to be a typing.Iterator doesn't implement that __iter__ method.

Contrary to those, the official Python tutorial section on iterators only mentions the need for a __next__ method:

The function returns an iterator object that defines the method __next__() which accesses elements in the container one at a time

So my question is: do Python iterators formally need to be iterables on their own? I personally don't understand why this should be true, and why we can't fully separate the requirements for an Iterable and an Iterator.

I am trying to implement an iterable proxy for a web resource (lazily fetched images).

Firstly, I did (returning ids, in production those will be image buffers)

I have been trying to code a neural network from scratch and have watched a couple of videos to see how it is implemented.

So I came across this guide that builds a simple neural network in Python.

I'm running a C++ application which tries to run python using the https://docs.python.org/3.5/extending/embedding.html function calls. This is the error that the application error message pipes are giving me.

class 'ImportError': Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. If you're working with a numpy git repo, try git clean -xdf (removes all files not under version control). Otherwise reinstall numpy.

Original error was: /usr/local/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so: undefined symbol: PyExc_UserWarning

I'm quite puzzled as this only occurs when embedding Python in C++ as the import works when I use it through the interpreter. I'm more interested in an answer that adds to my understanding than a quick do this or do that fix. I list some system/problem information below, and some other questions that I'm considering posting about the same topic. Any guidance is appreciated!

System/Problem information:

• Ubuntu 16.04, 64 bit
• Compiled Python 3.5.5 with enabled-shared
• numpy import works in the interpreter (python3.exe, and python3.5.exe)
• I have made sure that the PySys_SetPath() sets the same sys.path as the output from the interpreter: import sys, sys.path
• I can import other modules like PIL, and datetimeutil; however, numpy and pandas are not importable (pandas uses numpy or seems to)
• The embedded Python uses the following commands: Py_Import_Import(), Py_Initialize() (I made sure. It is only called once.), etc., but it does not get a global lock on the interpreter.
• The application is built with a CMake build system which compiles to MakeFiles for my system.
• Installed numpy-1.14.2 using pip 9.0.0 using the pip3.5 install numpy command
• The python script that causes this error has one line: import numpy...
• I do not have a .zip file that I'm importing files from.
• The .exe used by the Python embedded in the C++ is located at /usr/local/bin/python3 (used Py_GetProgramName() to determine this). This .exe is linked to the libpython3.5m.so.1.0, and the missing symbol lives in libpython3.5m.so.1.0 (ran nm)

• ldd on multiarray.cpython-35m-x86_64-linux-gnu.so shows:

  ldd multiarray.cpython-35m-x86_64-linux-gnu.so

  linux-vdso.so.1 => (0x00007ffd9e36b000)

  libopenblasp-r0-39a31c03.2.18.so => /usr/local/lib/python3.5/site-packages/numpy/core/./../.libs/libopenblasp-r0-39a31c03.2.18.so (0x00007fdbe149b000)

  libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdbe1192000)

  libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdbe0f75000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdbe0bab000) /lib64/ld-linux-x86-64.so.2 (0x00007fdbe3ed5000)

  libgfortran-ed201abd.so.3.0.0 => /usr/local/lib/python3.5/site-packages/numpy/core/./../.libs/libgfortran-ed201abd.so.3.0.0 (0x00007fdbe08b1000)

I could/might try reinstalling numpy through different means, but I'm having trouble tracking why that might work.

At this point, I'm assuming some hole in my knowledge exists. I have looked at a lot of similar posts regarding not being able to import the multiarray component and numpy when embedding Python in C++; however, either none of them match my specific case or as I stated there exists a hole. Here are a list of sub-questions that I will probably be asking if no one sees anything in this setup that is obviously concerning. I'll probably update the questions with links when/if I ask them (After I polish them).

• How does the numpy multiarray.so link to the pythonX.X.so for symbol resolution? The ldd does not seem to suggest that it ever does. Asked this question at this link
• CMake Question non-related issue resolved in this question asked on 4/12/18 and answered on 4/16/18.
• Setting PYTHONPATH in .bashrc does not seem to update what Py_GetPath() returns, I had to add in the site-packages for imports through a different methodology to sys.path. It may only update the bash script environment variable which doesn't effect the C++.

I'm not asking for an answer for the above question list at this point, rather I'm giving more clues to where my gap in knowledge may be.

Thank you for taking time from your day to read this question. Any help is appreciated.

Well, I found a work around, and I'm currently using it. Dunes question started making me think more closely about undefined symbols and how it could be a linker/compiler error or that the numpy import always expects an environment with those symbols already loaded into memory. This got me trying to install different versions of numpy to see if any of the older versions made a difference. They did not, but it did make the error thrown to be slightly different. When I googled that, this question appeared. The accepted answer gave me a work around by adding these two lines to the pythonInterface.cpp:

• #include
• dlopen("libpython3.5m.so.1.0", RTLD_LAZY | RTLD_GLOBAL)

These commands add the shared library to be loaded in and available to the cpython.multiarray.so.

This is not an ideal solution as pointing to a specific .so which may be different from machine to machine. It resolves the issue for now, but it also could lead to errors where mismatches of shared libraries can occur during the python call process if the linked library to the pythonInterface.so changes, and this line does not get updated. I believe a better answer can be achieved if this sub-question is answered, so I'm currently holding out on submitting or accepting an answer until then. Thanks!

I am trying to implement the backpropagation algorithm to show how a two layered neural network can be used to behave as the XOR logic gate. I followed this tutorial here.

After running, I expect the output to follow the XOR logic truth table:

I am currently learning about how to code neural networks in numpy/python. I used the code from this tutorial and tried to adapt it to make an importable module. However, when i tried using my own dataset. It threw a numpy error ValueError: shapes (1,3) and (1,1) not aligned: 3 (dim 1) != 1 (dim 0).

I have already tried reshaping all of the matrices from (x,) to (x,1) but with no success. After a bit of reading around, transposing the arrays was also meant to fix the issue, but i tried that as well and no success there either.

Here is the module (called hidden_net):

I have a simple MYSQL pod sitting behind a MYSQL service.

Additionally I have another pod that is running a python process that is trying to connect to the MYSQL pod.

If I try connecting to the IP address of the MYSQL pod manually from the python pod, everything is A-OK. However if I try connecting to the MYSQL service then I get an error that I can't connect to MYSQL.

I want to create a simple neural network and in my studies, I reached a concept called Gradient Descent. That says:

Imagine that you had a red ball inside of a rounded bucket. Imagine further that the red ball is trying to find the bottom of the bucket. This is optimization.

I use this tutorial:

http://iamtrask.github.io/2015/07/27/python-network-part2/

But I can't understand when optimization happens. When gradient descent happens and most importantly, What is the relation with the rounded bucket example?

This website has another tutorial(basic method):

http://iamtrask.github.io/2015/07/12/basic-python-network/

According to this website, This one is not gradient descent but the second tutorial is the same is first one and it called gradient descent method. I can't understand these two differences.

I realised I still don't quite understand how to implement an iterator class. So a class where one can "for-loop" over some of its contents.

I have looked at these answers (I still don't get it):

What exactly are Python's iterator, iterable, and iteration protocols?

Build a Basic Python Iterator

As far as I understand, an iterable is one that implements __iter__ and returns an iterator which is something that has __next__ implemented. From this I somehow understood that if I want my class to be an iterator, and iterable. I must define __iter__ to return self, and have __next__ defined. Am I wrong so far?

Here is a scaffold for my class: