I need raw buffers of very big size (let say more MB than KB) as light as possible. I would like to have it keep the data in dynamic area, not necessarily it shall be grow-able, I will set the size at construction.
I thought about std::vector<unsigned char> but:
std::vector<unsigned char> a(VERY_BIG_SIZE);
has the data initialized to 0 - I don't need this, I don't want to pay for it... This is embedded system already of high usage of CPU/RAM and I want to benefit from the fact that memory allocated but not used is only virtually allocated (I mean it takes some address space but it is not mapped to real memory unless used).
I thought also about:
std::vector<unsigned char> a;
a.reserve(VERY_BIG_SIZE);
std::copy(someData..., a.begin());
it surprisingly works as expected (I guess this UB somehow) - and the a whole memory is not initialized, but as you already noticed, I cannot copy a to other vector because (a.begin()== a.end())....
I need to explain somehow why I did not do push_back(...)/insert(a.end(), ...)/assign(...) in my second approach.
I already have template class Buffer with constant size:
template <size_t SIZE>
class Buffer {
public:
// this "dirty" solution works, but I would need to add copy stuff by myself...
Buffer() { data.reserve(SIZE); }
// this "correct" solution is too heavy:
Buffer() : data(SIZE) { }
unsigned char* data() const { return &data[0]; }
private:
std::vector<unsigned char> data;
// unsigned char data[SIZE]; // this is not an option because it is not dynamic memory
};
Is there anything I can put in private part of Buffer that will take care of memory management and it will be copyable and not initialized... And memory would be dynamically allocated (like std::Vector does) that is why unsigned char data[SIZE]; is not an option.
Any ideas?
Std vector of a struct that is packed to 1 byte and whose default constructor leaves then contents uninitialized (ie does nothing) should work.
Note that you can do mass inserts into an empty vector, and the only initialization will be from the incoming data. Ie, vect.insert( vect.end(), random_access_iterator_begin, blah_end ) might have the performance you want.
You are almost on the right track with vector::reserve.
I recommend using reserve in combination with vector::assign and vector::push_back. That does exactly what you want.
std::vector<unsigned char> a;
a.reserve(VERY_BIG); // check capacity to see how much memory was reserved
std::copy(someData.begin(), someData.end(),
std::back_inserter(a)); // allocation free copy
// or
a.assign(someData.begin(), someData.end());
After your update, I'd implement my own buffer.
template<std::size_t Size>
class buffer {
public:
buffer() : b_(new unsigned char[Size]) {}
buffer(const buffer& other) {
std::copy(other.begin(), other.end(), this->begin());
}
buffer& operator=(const buffer& other) {
// intentionally no protection against self assignment
std::copy(other.begin(), other.end(), this->begin());
return *this;
}
~buffer() { delete [] b_; }
// move operations, if desired
unsigned char* begin() { return b_; }
unsigned char* end() { return b_ + Size; }
// const versions...
private:
unsigned char* b_;
};
This should have all your desired characteristics (heap allocated,
deep copyable, uninitialized on creation).
All of the standard containers initialize all the memory asked for. Using a std::vector<unsigned char> with reserved memory seems to be the option, though: You can always add elements using
std::vector<unsigned char> a;
a.reserve(VERY_BIG_SIZE);
a.insert(a.end(), new_data.begin(), new_data.end());
Puts the data directly into the appropriate location without extra initialization. Of course, this means that the data you actually did store in your container is contiguous at the start of the container. If you need to access random positions in your big buffer and want to be able to only copy the corresponding sections things become a lot more involved and there isn't anything in the standard C++ library readily doing this.
BTW, since you linked to this question from an approach using a stream buffer to write data into an area: it would be possible to combine the approach but it would be more involved than using the vector's storage as the stream's output buffer: Since you'd need to track the size of the vector you'd need to make the available buffer empty, having it incremented for each character. This would technically work but would be slow. Instead, you'd give the stream buffer a buffer and transfer it to the end of your std::vector<unsigned char> upon overflow() or when pubsync()ing the stream buffer (which is what std::flush triggers).
You could always do unsigned char *a = new unsigned char[n];.
