How to access this Python Dict elements?

Question

I have a web scraper script that creates a nested dictionary from the pages it scrapes.
I don't seem to be able to reference the nested dict.

The basic format is a tuple for the main key, and the value is the nested dict:
{ ("Moto X4", "Verizon") : { "Carrier": "Verizon", "Listings" : "27 available", "Screen" : "5.5 in." } }

I can iterate through the dict, but all I seem to get is the key, or the contents of the key as items or as a set, or as a list. I've tried using box notation (i.e. ...["Screen"] ), etc.

Any thoughts on what I'm doing wrong?

Ultimately, I can change the data structure back to a list, this is more of an exercise in scraping, with a side of "my_dict" data structures.

Answer 1

Sometimes it is a good idea to start again completely.
I occasionally delete all my previous attempts to force my mind to NOT refer to what I did before.
Example of Named Tuple attached.
Good luck

https://code.sololearn.com/cYtN0UazY6Qr/?ref=app

Answer 2

Does this help.

https://code.sololearn.com/cT2dC2KtqwsH/?ref=app

Answer 3

Instead of a dictionary, why don't you look at panda, or named tuples.
They might be able to provide the functions you want

Answer 4

I am still learning about panda myself, but attached is a little example
https://code.sololearn.com/cxBcbws6Ll20/?ref=app

Answer 5

Would you like an example of a named tuple to play with also?

Answer 6

That looks very promising!
I may give it a rewrite tomorrow x
Happy New Year!

Answer 7

Back to your original concept - accessing nested dict values through iteration.
See if this works for you
https://code.sololearn.com/cEWRlBSsNSM5/?ref=app

Answer 8

Unfortunately, it doesn't help me.
It works with literals, but I can't get it as iterables.
I may have provided a bad data example, or I'm just not "getting" iteration and dict references.

Here's the ugly truth:
https://code.sololearn.com/cYjL3aZ7zPZ5

I thought that SL had the Python Requests and BeautifulSoup libraries, but either they don't, or I may not know how to reference them.

The end result is, I can't run this on SL, as is, and when I run locally, I can't get my dict references to work.

If I resort to an earlier version with lists, I can copy/paste or export that and still use it, but I really want to figure out how to dig my nested dict out, if possible.

Answer 9

Sololearn does not import many modules.
Try to resolve this on another IDE.
Good luck, hopefully a guru will be able to help you

Answer 10

Updated notes:
SL has the library:
urllib.request

I may try to re-write it to that, but that still leaves the problem of missing BeautifulSoup, before I can work on my iteration and dictionaries.

I have created a chopped version, starting from the resulting Dict from the scraping code.

https://code.sololearn.com/ct4Y0XviMt7E

Also, I just realized that I may be over-thinking this.
Is there a better way to construct the dictionary, that maintains the multiple phone/carrier capability?

i.e.,
my_dict =
{ ('Moto Z2 Force', 'Verizon'): {
'Carrier': 'Verizon',
'Screen': '5.5"',
'Price': 'Starts at $78',
'Listings': '13 available',
'Item': 6
},
('Moto Z2 Force', 'T-Mobile'): {
'Carrier': 'T-Mobile',
'Screen': '5.5"',
'Price': 'Starts at $70',
'Listings': '8 available',
'Item': 9
},
('Moto G6', 'Verizon'): {
'Carrier': 'Verizon',
'Screen': '5.7"',
'Price': 'Starts at $89',
'Listings': '5 available',
'Item': 13
},
('Moto G6', 'Unlocked'): {
'Carrier': 'Unlocked',
'Screen': '5.7"',
'Price': 'Starts at $85',
'Listings': '3 available',
'Item': 17
},
('Moto E4', 'T-Mobile'): {
'Carrier': 'T-Mobile',
'Screen': '5.0"',
'Price': 'Starts at $50',
'Listings': '2 available',
'Item': 18
},
('Moto E4', 'Unlocked'): {
'Carrier': 'Unlocked',
'Screen': '5.0"',
'Price': 'Starts at $41',
'Listings': '2 available',
'Item': 19
}
}

Answer 11

Rik Wittkopp , I'll look at named tuples.and see what I can do with those.
What does Pandas provide? I thought that it was a large numeric analysis library.

Answer 12

I will try this tomorrow... I've spent a couple hours tonight trying understand enough to get a structure together, but I'm just not successful yet.

The funny thing is, I could just manually scrape and format on a couple of minutes, but I really want to understand the data structure capabilities.

Answer 13

@Rik_Wittkopp,
Please. I know I need help focusing down on this - I know that I start searching and when I can't narrow down, I expand. I just checked my Firefox tab load... 5 windows, 418 tabs. I have a whole window dedicated to Python, Pandas, Data, and exercises... nothing that I can apply, tho'.

This is why I keep drifting further from the target - looking for shortcuts.

My choice of data structures is probably off... I think that my earliest attempt with just individual variables, then converting into Dicts - which I felt I had a handle on - has caused me to drift. Each time I got "kind of" an answer, I changed code to head in that direction, until I don't have clean code to fall back on.

Answer 14

Rik,
I'm up past my bedtime :)
It's not just New Year's Eve, I just did a couple hours of Skyrim!

I don't think the last example set will work for the reason of needing both the model and carrier to get a unique key to capture the mixed cases: Motorola G6, can be Verizon or Unlocked: Apple devices can be found on just about any carrier, as Google, I think.

That's how I wound up with a tuple for a key, although I probably could have gone with a concatenated string, I think. Maybe the problem was, I don't know if I can iterate through a dictionary using a tuple for a key.

I'll give it another try when I am fresh later.

When I get this figured, I think it will be broken into several examples for SL.

Answer 15

Good night buddy.
I like your determination